Image Display Apparatus, Video Signal Processor, and Video Signal Processing Method

ABSTRACT

The present invention provides an image display apparatus capable of suppressing deterioration in picture quality resulting from precision of detection of a motion vector at the time of performing a predetermined video signal process to improve picture quality. In consideration of reliability in detection of a motion vector mv by a motion vector detector  44 , a video signal process in an interpolation section  45 , an imaging blur suppression processor  13 , and an overdrive processor  10  is performed. Concretely, the video signal process is performed so that a degree of the video signal process rises as reliability increases and, on the other hand, a degree of the video signal process falls off as the reliability decreases. In the case of performing the video signal process using a motion vector, even when a motion vector lies out of a motion vector search range (block matching range), the video signal process according to the motion vector detection precision can be performed.

TECHNICAL FIELD

The present invention relates to a video signal processor and a videosignal processing method for performing a predetermined video signalprocessing using motion compensation and to an image display apparatushaving such a video signal processor.

BACKGROUND ART

As one of video signal processes for improving picture quality in atelevision receiver, a DVD player, and the like there is frame rateconversion using motion compensation.

The principle of frame rate conversion will be described using FIGS. 1to 3 with respect to a video signal captured by a camera for televisionbroadcast (hereinbelow, referred to as a camera signal) and a videosignal obtained by telecine converting a film to a television system(hereinbelow, called film signal or cinema signal).

FIG. 1( a) shows original frames A, B, C, and D of a camera signal of anNTSC system. In the case of converting the frame rate of the camerasignal to 120 Hz, as shown in FIG. 1( b), an interpolation frame isadded at a timing of every 1/120 sec between neighboring original frames(between the frames A and B, between the frames B and C, and between theframes C and D).

FIG. 2( a) shows original frames A, B, C, and D of a film signal whichis telecine converted (2:2 pulldown) to the PAL system. Each of theoriginal frames is repeated twice. In the case of converting the framerate of the 2:2 pulldown film signal to 100 Hz, as shown in FIG. 2( b),three interpolation frames are added at 1/100 sec intervals betweenoriginal frames neighboring at 25 Hz cycles (between the frames A and B,between frames B and C, and between frames C and D).

FIG. 3( a) shows original frames A, B, and C of a film signal which istelecine converted (3:2 pulldown) to the NTSC system. The odd-numberedoriginal frames A and C are repeated three times, and the even-numberedoriginal frame B is repeated twice. In the case of converting the framerate of the 3:2 pulldown film signal to 120 Hz, as shown in FIG. 3( b),four interpolation frames are added at 1/120 sec intervals betweenoriginal frames neighboring at 24 Hz cycles (between the frames A and Band between the frames B and C).

Each of the interpolation frames is generated by interpolating videoimages of an earlier original frame and a following original frame. Theinterpolation is performed by a method of calculating addresses ofpixels of the earlier original frame and the following original frameused for calculating pixel values of an interpolation frame on the basisof parameters of an interpolation position of a video image in eachinterpolation frame and motion vectors between the earlier originalframe and the following original frame, and then placing weights to thepixel values of the addresses in accordance with interpolationpositions.

The frame rate conversion produces an effect of eliminating a motionblur in a camera signal and an effect of reducing a judder (unsmoothnessof motion in a video image) in a film signal.

FIGS. 1 to 3 also show interpolation positions of video images in theinterpolation frames in the conventional frame rate conversion. As shownin FIG. 1( b), the interpolation position of a video image in theinterpolation frames, at the time of converting the frame rate of theNTSC camera signal to 120 Hz, is conventionally set to a positionobtained by equally dividing the magnitude of the motion of a videoimage between the earlier original frame and the following originalframe (the magnitude determined by motion vector detection) to twoportions, that is, a position of 50% of the magnitude of the motion.

As shown in FIG. 2( b), the interpolation positions of video images inthree interpolation frames at the time of converting the frame rate ofthe 2:2 pulldown film signal to 100 Hz are conventionally set topositions obtained by equally dividing the magnitude of video imagemotion between the earlier original frame and the following originalframe to four portions, that is, positions of 25%, 50%, and 75% of themagnitude of the motion.

As shown in FIG. 3( b), the interpolation positions of video images infour interpolation frames at the time of converting the frame rate ofthe 3:2 pulldown film signal to 120 Hz are conventionally set topositions obtained by equally dividing the magnitude of the motion of avideo image between the earlier original frame and the followingoriginal frame to five portions, that is, positions of 20%, 40%, 60%,and 80% of the magnitude of the motion.

FIG. 4 is a diagram showing examples of video images of the 3:2 pulldownfilm signal subjected to the frame rate conversion in the interpolationpositions in the related art. The video image of an airplane movesbetween neighboring original frames A and B. In four interpolationframes, video images of the airplane are interpolated in positionsobtained by equally dividing the magnitude of the motion into fiveportions.

In addition, for example, a technique related to such frame rateconversion is proposed in Patent document 1.

Patent document 1: Japanese Unexamined Patent Application PublicationNo. 2003-189257

DISCLOSURE OF INVENTION

As described above, in the frame rate conversion using motioncompensation, conventionally, interpolation positions of a video imagein interpolation frames are set to positions obtained by evenly dividingthe magnitude of a motion of the video image between the earlieroriginal frame and the following original frame.

However, in the case of a film signal, when interpolation is performedin the interpolation positions obtained by evenly dividing the magnitudeof the motion of the video image between original frames as exemplifiedin FIG. 4, a judder is largely reduced, and the motion of the videoimage becomes very smooth. As a result, a person accustomed to a judderin a film signal gets an impression that the taste of the film signal islost.

Further, in the frame rate conversion using motion compensation, in thecase where the motion of a video image between neighboring originalframes becomes very fast, the motion vector lies out of a motion vectorsearch range (block matching range), and a large judder occurs. In sucha case, there is a problem that since a large judder suddenly occurswhile the user is watching a video image in which motion is very smooth,the user feels discomfort.

In addition, hitherto, to make motion of a video image at the time ofconverting the frame rate of a film signal (cinema signal) smoother, atechnique of shifting a pixel position of a field after the frame rateconversion in the direction of a motion vector has been proposed (referto Patent Document 1). However, a technique of weakening the degree ofreduction of a judder while reducing the judder at the time ofconverting the frame rate of a film signal is not proposed.

By the way, in the case of performing a video signal process to improvethe picture quality such as the frame rate conversion by using a motionvector, when the motion vector lies out of the motion vector searchrange (block matching range) as above, the motion vector may not be ableto be detected well. In such a case, if the motion vector is used as itis, the video signal process is not performed well. Another issue arisessuch that picture quality deteriorates.

Further, in the case of displaying the video signal subjected to thevideo signal process on a fixed pixel (hold) type display apparatus suchas a liquid crystal display, another issue occurs such that a so-calledhold blur occurs due to its configuration. It is demanded to reduce thehold blur as much as possible. Since viewability of such a hold blurvaries according to circumstances, an improvement method according tocircumstances is desired.

The present invention has been achieved in view of such issues, and itsfirst object is to provide an image display apparatus, a video signalprocessor, and a video signal processing method capable of weakening thedegree of reduction of a judder while reducing the judder at the time ofconverting frame rate of a film signal (cinema signal) using motioncompensation.

A second object of the present invention is to provide an image displayapparatus, a video signal processor, and a video signal processingmethod capable of suppressing deterioration in picture quality due todetection precision of a motion vector at the time of performing apredetermined video signal process to improve picture quality.

Further, a third object of the present invention is to provide an imagedisplay apparatus capable of reducing a hold blur in accordance withcircumstances.

An image display apparatus of the invention includes: motion vectordetecting means for detecting a motion vector in a plurality of originalframes along a time base; video signal processing means for performing,by using the detected motion vector, a predetermined video signalprocess to improve picture quality on the plurality of original frames;and display means for displaying a video image on the basis of a videosignal subjected to the video signal process. Further, the video signalprocessing means performs the video signal process so that a degree ofthe video signal process rises as reliability in detection of the motionvector by the motion vector detecting means increases and, on the otherhand, a degree of the video signal process falls off as the reliabilitydecreases.

A video signal processor of the invention includes: motion vectordetecting means for detecting a motion vector in a plurality of originalframes along a time base; and video signal processing means forperforming, by using the detected motion vector, a predetermined videosignal process to improve picture quality on the plurality of originalframes. Further, the video signal processing means performs the videosignal process so that a degree of the video signal process rises asreliability in detection of the motion vector by the motion vectordetecting means increases and, on the other hand, a degree of the videosignal process falls off as the reliability decreases.

A video signal processing method of the present invention includes thesteps of: detecting a motion vector in a plurality of original framesalong a time base; and performing, by using the detected motion vector,a predetermined video signal process to improve picture quality on theplurality of original frames, while performing the video signal processso that a degree of the video signal process rises as reliability indetection of the motion vector increases and, on the other hand, adegree of the video signal process falls off as the reliabilitydecreases.

In the image display apparatus, the video signal processor, and thevideo signal processing method of the invention, a motion vector isdetected in a plurality of original frames along a time base, and apredetermined video signal process to improve picture quality isperformed on the plurality of original frames by using the detectedmotion vector. In such a video signal process, the video signal processis performed so that a degree of the video signal process rises asreliability in detection of a motion vector increases and, on the otherhand, a degree of the video signal process falls off as the reliabilitydecreases. Consequently, for example, also in the case where a motionvector lies out of a motion vector search range (block matching range),a video signal process according to motion vector detection precisioncan be performed.

In the image display apparatus, the video signal processor, and thevideo signal processing method of the invention, the video signalprocess is performed so that a degree of the video signal process risesas reliability in detection of a motion vector increases and, on theother hand, a degree of the video signal process falls off as thereliability decreases. Accordingly, in the case of performing apredetermined video signal process to improve picture quality by usingthe motion vector, a video signal process according to motion vectordetection precision can be performed. Therefore, at the time ofperforming the predetermined video signal process to improve the picturequality, deterioration in picture quality according to the motion vectordetection precision can be suppressed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 Diagrams showing the principle of frame rate conversion of acamera signal and interpolation positions in related art.

FIG. 2 Diagrams showing the principle of frame rate conversion of a filmsignal and interpolation positions in related art.

FIG. 3 Diagrams showing the principle of frame rate conversion of a filmsignal and interpolation positions in related art.

FIG. 4 A diagram exemplifying video images of the film signal subjectedto frame rate conversion in the interpolation positions in related art.

FIG. 5 A block diagram showing an example of a circuit configuration ofa video signal processor according to a first embodiment of theinvention.

FIG. 6 A diagram showing the principle of address calculation byinterpolator.

FIG. 7 A diagram showing interpolation position parameters supplied froma CPU.

FIG. 8 A diagram showing values of the interpolation position parametersin the case of a 3:2 pulldown film signal.

FIG. 9 A diagram showing values of the interpolation position parametersin the case of a 2:2 pulldown film signal.

FIG. 10 Diagrams showing video images of film signals subjected to theframe rate conversion using the apparatus of FIG. 5.

FIG. 11 A block diagram showing an example of the circuit configurationof a video signal processor according to a modified example of the firstembodiment.

FIG. 12 Diagrams showing frame rate conversion to 240 Hz of a camerasignal.

FIG. 13 A block diagram showing an example of the configuration of avideo signal processor according to a second embodiment of the presentinvention.

FIG. 14 A diagram showing an example of a frequency characteristic of ablur in an image formed on the retina of a human.

FIG. 15 A flowchart explaining an example of an image process executedby the video signal processor of FIG. 13.

FIG. 16 A diagram showing an example of the frequency characteristic ofan imaging blur according to a travel vector (travel speed, motionvector).

FIG. 17 A block diagram showing an example of a functional configurationof an imaging blur suppression processor in the video signal processorof FIG. 13.

FIG. 18 A block diagram showing an example of the functionalconfiguration of a high frequency component removing unit in the imagingblur suppression processor of FIG. 17.

FIG. 19 A diagram showing an example of the characteristic of ahigh-frequency limiter in the high frequency component removing unit ofFIG. 18.

FIG. 20 A block diagram showing an example of the functionalconfiguration of a filter unit in the imaging blur suppression processorof FIG. 17.

FIG. 21 A block diagram showing an example of the functionalconfiguration of a gain controller in the filter unit in FIG. 20.

FIG. 22 A diagram showing an example of the characteristic of anadjustment amount determining unit in the gain controller in FIG. 21.

FIG. 23 A block diagram showing an example of the functionalconfiguration of an imaging blur compensating unit in the imaging blursuppression processor of FIG. 17.

FIG. 24 A block diagram showing an example of the functionalconfiguration of an ALTI unit in the imaging blur compensating unit inFIG. 23.

FIG. 25 A diagram illustrating an example of an object to be processedof the ALTI unit of FIG. 24 to explain a method of correcting a pixelvalue in the case of computing an average of pixel values of a group ofpixels arranged successively on the right side of a target pixel.

FIG. 26 A diagram for supplementarily explaining the pixel valuecorrecting method in the case of computing the average of pixel valuesof a group of pixels arranged successively on the right side of thetarget pixel.

FIG. 27 A flowchart explaining an example of processes of the ALTI unitof FIG. 24.

FIG. 28 A diagram showing an example of the characteristic of anadjustment amount calculator in the ALTI unit of FIG. 24.

FIG. 29 A block diagram showing another example different from FIG. 12of the functional configuration of the ALTI unit in the imaging blurcompensating unit of FIG. 23.

FIG. 30 A block diagram showing an example of the functionalconfiguration of the gain controller in the imaging blur compensatingunit in FIG. 23.

FIG. 31 A diagram showing an example of the characteristic of theadjustment amount determining unit in the gain adjusting unit in FIG.30.

FIG. 32 A block diagram showing an example different from FIG. 17 of thefunctional configuration of the imaging blur suppression processor inthe video signal processor of FIG. 13.

FIG. 33 A block diagram showing an example different from FIGS. 17 and32 of the functional configuration of the imaging blur suppressionprocessor in the video signal processor of FIG. 13.

FIG. 34 A diagram illustrating shutter speed of a camera and thecharacteristic of an imaging blur.

FIG. 35 A block diagram showing an example different from FIG. 13 of theconfiguration of a part of a video signal processor according to thesecond embodiment.

FIG. 36 A block diagram showing an example different from FIGS. 13 and35 of the configuration of a part of the video signal processor of thesecond embodiment.

FIG. 37 A block diagram showing an example different from FIGS. 13, 35,and 36 of the configuration of a part of the video signal processor ofthe second embodiment.

FIG. 38 A block diagram showing an example different from FIGS. 13, 35,36, and 37 of the configuration of a part of the video signal processorof the second embodiment.

FIG. 39 A block diagram showing an example different from FIGS. 17, 32,and 33, of the functional configuration of the imaging blur suppressionprocessor in the video signal processor of FIG. 13.

FIG. 40 A block diagram showing an example different from FIGS. 17, 32,33, and 39, of the functional configuration of the imaging blursuppression processor in the video signal processor of FIG. 13.

FIG. 41 A block diagram showing an example different from FIGS. 17, 32,33, 39, and 40, of the functional configuration of the imaging blursuppression processor in the video signal processor of FIG. 13.

FIG. 42 A block diagram showing the configuration of a video signalprocessor according to a modified example of the second embodiment.

FIG. 43 A block diagram showing an example of the configuration of avideo signal processor according to a modified example of a thirdembodiment of the invention.

FIG. 44 A diagram showing an example of the relation between thepresence/absence of detection of a motion vector and reliability.

FIG. 45 A timing waveform chart showing an example of the relationbetween the presence/absence of detection of a motion vector andreliability.

FIG. 46 A timing chart showing an example of a change in a gainmultiplied with a motion vector according to reliability.

FIG. 47 A timing chart showing another example of a change in a gainmultiplied with a motion vector according to reliability.

FIG. 48 A block diagram showing an example of the configuration of animage display apparatus according to a fourth embodiment of the presentinvention.

FIG. 49 A timing chart showing an example of a black inserting process(blinking process) on the frame unit basis by a backlight driving unitillustrated in FIG. 48.

FIG. 50 A timing chart showing an example of the black inserting process(blinking process) on the black insertion line unit basis by thebacklight driving unit illustrated in FIG. 48.

FIG. 51 A timing chart showing an example of the black inserting process(blinking process) in combination of the black insertion line unit basisand the frame unit basis by the backlight driving unit illustrated inFIG. 48.

FIG. 52 A timing chart showing an example of a black insertion ratio inthe black inserting process on the frame unit basis.

FIG. 53 A timing chart showing another example of the black insertionratio in the black inserting process on the frame unit basis.

FIG. 54 A timing chart showing an example of the black insertion ratioin the black inserting process in combination of the black insertionline unit basis and the frame unit basis.

FIG. 55 A timing chart showing another example of the black insertionratio in the black inserting process in combination of the blackinsertion line unit basis and the frame unit basis.

FIG. 56 A characteristic diagram showing an example of a luminancehistogram distribution of an entire screen.

FIG. 57 A block diagram showing an example of the configuration of animage display apparatus according to a modified example of the fourthembodiment.

FIG. 58 A block diagram showing an example of a hardware configurationof all or part of a video signal processor to which the presentinvention is applied.

BEST MODE(S) FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described in detailhereinbelow with reference to the drawings.

First Embodiment

FIG. 5 is a block diagram showing an example of the circuitconfiguration of a video signal processor (video signal processor 4)according to a first embodiment of the present invention. The videosignal processor 4 is built in a television receiver. A digitalcomponent signal YUV subjected to processes such as tuning and decodingby a tuner, a decoder, and the like which are not shown is supplied tothe video signal processor 4.

The digital component signal YUV supplied to the video signal processor4 is input to a preprocessor 41 and sequentially written to a memory 43via a memory controller 42.

The preprocessor 41 performs a process of separating a luminance signalY from the digital component signal YUV. The luminance signal Yseparated by the preprocessor 41 is also sequentially written in thememory 43 via the memory controller 42.

The luminance signal Y written in the memory 43 is sequentially read bythe memory controller 42 (as shown in FIGS. 2 and 3, in the case of afilm signal in which the same original frame is repeated twice or threetimes, the same original frame is read only once) and is sent to amotion vector detector 44. The motion vector detector 44 performs amotion vector detecting process by block matching using the luminancesignal Y of the present frame and the luminance signals Y of theimmediately preceding and subsequent frames.

A motion vector mv of each of the frames detected by the motion vectordetector 44 is written in the memory 43 via the memory controller 42.After that, the motion vector mv is read from the memory 43 and sentagain to the motion vector detector 44 so as to be referred to in motionvector detection of the following frame.

Further, the memory controller 42 reads the digital component signalsYUV written in the memory 43 at double speed, in two series whiledeviating the signals from each other by one frame (in the case of afilm signal in which the same original frame is repeated twice or threetimes as shown in FIGS. 2 and 3, the same original frame is read onlyonce). Further, the memory controller 42 reads the motion vector mvindicative of motion between the two frames at double speed. The twoseries of digital component signals 2YUV and the motion vector mv readin such a manner are transmitted to an interpolation section 45.

The interpolation section 45 is provided with two series of search rangememories 451 and 452 and an interpolator 453. One of the two series ofdigital component signals 2YUV from the memory controller 42 is writtento the search range memory 451 and the other is written to the searchrange memory 452. The motion vector mv from the memory controller 42 isinput to the interpolator 453.

Further, from a CPU 46 in the television receiver, an interpolationposition parameter Relpos indicative of the interpolation position of avideo image in an interpolation frame is supplied to the interpolationsection 45 via an I²C bus 40 as a serial bus and a decoder 47 forparallel converting the serial signal (the details of the interpolationposition parameter Relpos will be described later). The interpolationposition parameter Relpos is also input to the interpolator 453.

On the basis of the motion vector mv and the interpolation positionparameter Relpos, the interpolator 453 calculates addresses of pixels inoriginal frames in the search range memories 451 and 452 used forcalculating pixel values of an interpolation frame.

FIG. 6 is a diagram conceptually showing the principle of the addresscalculation. n−1 indicates, in the vertical axis direction, an address(the position in the x direction and the y direction on the screen) ofeach of pixels of an original frame which comes first out of the twooriginal frames deviated by one frame which are written in the searchrange memories 451 and 452. n indicates, in the vertical axis direction,an address of each of pixels of the later original frame out of the twooriginal frames.

i indicates the address of each of pixels in the interpolation frame inthe vertical axis direction. The horizontal axis indicates time andshows timing of the interpolation frame i between the original framesn−1 and n (herein as an example, the timing corresponding to theinterpolation frame in the center out of the three interpolation framesin FIG. 2( b)). Relpos indicates an interpolation position parametersupplied for generation of the interpolation frame i.

mv(x,y)int shows the motion vector mv between the original frames n−1and n with respect to the address (x, y) of a pixel which is beinggenerated at present (called reference pixel) in each of pixels of theinterpolation frame i. zeroPelPrev(x,y) indicates the value of a pixelin the reference address (x, y) in the original frame n−1.zeroPelSucc(x, y) indicates the value of a pixel in the referenceaddress (x, y) in the original frame n.

The interpolator 453 obtains the addresses of the pixels in the originalframes n−1 and n used for calculating the pixel value of the referenceaddress (x, y) by the following formula (1) on the basis of thereference address (x, y), a component mvX in the x direction of themotion vector mv(x,y)int, a component mvY in the y direction of themotion vector mv(x,y)int, and the interpolation position parameterRelpos.

[Mathematical Formula 1]

Address of pixel in original frame n−1=(x+mvX·Relpos,y+mvY·Relpos)

Address of pixel in original framen=(x−mvX·(1−Relpos),y−mvY·(1−Relpos))  (1)

The interpolator 453 sends the addresses obtained as described above tothe search range memories 451 and 452 and reads pixel values prev andsucc of the addresses. Then using the pixel values prev and succ and theinterpolation position parameter Relpos, a pixel value Out of thereference address (x, y) of the interpolation frame i is calculated bythe following formula (2).

[Mathematical Formula 2]

Out=prev·(1−Relpos)+succ·Relpos  (2)

By executing such calculation sequentially on each of pixels of theinterpolation frame i (sequentially updating the value (x, y) of thereference address), the interpolation frame i is generated.

Next, the interpolation position parameter Relpos supplied from the CPU46 to the interpolation section 45 will now be described. FIG. 7 is adiagram showing the interpolation position parameter Relpos suppliedfrom the CPU 46. In the case where a 2:2 pulldown film signal (refer toFIG. 2( a)) is supplied as the digital component signal YUV to the videosignal processor 4 of FIG. 5, the CPU 46 supplies parameters of fourphases Relpos_22_0, Relpos_22_1, Relpos_22_2, and Relpos_22_3 every1/100 sec (that is, at 1/25 sec cycles). Each of the parameters of thephases is made of six bits ([5:0] in the diagram expresses six bits).

Relpos_22_0 is a parameter for outputting the preceding one of the twooriginal frames deviated as it is from each other by one frame in thesearch range memories 451 and 452 from the interpolator 453. Relpos_22_1to Relpos_22_3 are parameters for generating three interpolation framesat 1/100 sec intervals as shown in FIG. 2( b) between the two originalframes.

In the case where the 2:2 pulldown film signal is supplied, the sameoriginal frame is held for 1/25 sec in the search range memories 451 and452 (FIG. 5). Then, during 1/25 sec, the interpolator 453 calculates aninterpolation frame by the above-mentioned formulae (1) and (2) on eachof the parameters of respective phases Relpos_22_0, Relpos_22_1,Relpose_22_2, and Relpos_22_3. By repeating the process at 1/25 seccycles, the 2:2 pulldown film signal is frame-rate-converted.

On the other hand, in the case where a 3:2 pulldown film signal (referto FIG. 3( a)) is supplied as the digital component signal YUV to thevideo signal processor 4 of FIG. 5, the CPU 46 supplies interpolationposition parameters of five phases Relpos_32_0, Relpos_32_1,Relpos_32_2, Relpos_32_3, and Relpos_32_4 every 1/120 sec (that is, at1/24 sec cycles).

Relpos_32_0 is a parameter for outputting the preceding one of the twooriginal frames deviated from each other by one frame in the searchrange memories 451 and 452 as it is from the interpolator 453.Relpos_32_1 to Relpos_32_4 are parameters for generating fourinterpolation frames at 1/120 sec intervals as shown in FIG. 3( b)between the two original frames.

In the case where the 3:2 pulldown film signal is supplied, the sameoriginal frame is held for 1/24 sec in the search range memories 451 and452. Then, during 1/24 sec, the interpolator 453 calculates aninterpolation frame by the above-mentioned formulae (1) and (2) on eachof the parameters of respective phases Relpos_32_0, Relpos_32_1,Relpose_32_2, Relpos_32_3, and Relpos_32_4. By repeating the process at1/24 sec cycles, the 3:2 pulldown film signal is frame-rate-converted.

The value of the interpolation position parameter Relpos is selected bythe user. Specifically, as shown in FIG. 5, a remote controller 400attached to the television receiver is provided with an interpolationposition adjustment button 401 for switching and selecting the value ofthe interpolation position parameter Relpos in three levels of “strong,medium, and weak”.

A signal indicative of the selection result by the interpolationposition adjustment button 401 is received from the remote controller400 by an infrared light receiving unit 48 in the television receiver.When the signal is transmitted to the CPU 46 via the I²C bus 40, the CPU46 sets the value of the interpolation position parameter Relposaccording to the selection result.

FIG. 8 is a diagram showing the values of the interpolation positionparameters Relpos set by the CPU 46 in accordance with the selectionresult of the interpolation position adjustment button 401 in the casewhere the 3:2 pulldown film signal is supplied. In the case where“strong” is selected by the interpolation position adjustment button401, the values of Relpos_32_0, Relpos_32_1, Relpos_32_2, Relpos_32_3,and Relpos_32_4 are set to 0, 0.2, 0.4, 0.6, and 0.8, respectively.

Since the value of the parameter Relpos_32_0 of the first phase is 0,the preceding one of the two original frames in the search rangememories 451 and 452 is output as it is from the interpolator 453 fromthe formulae (1) and (2).

Further, since the values of the parameters Relpos_32_1, Relpos_32_2,Relpos_32_3, and Relpos_32_4 of the second, third, fourth, and fifthphases change uniformly by 0.2 like 0.2, 0.4, 0.6 and 0.8, from theformulae (1) and (2), the interpolation positions of video images in thefour interpolation frames generated between the two original frames inthe search range memories 451 and 452 are the same as the interpolationpositions in related art shown in FIG. 3( b). The positions are obtainedby uniformly dividing the magnitude of a motion of a video image betweenthe two original frames into five portions, that is, positions of 20%,40%, 60%, and 80% of the magnitude of the motion.

In the case where “medium” is selected by the interpolation positionadjustment button 401, the values of Relpos_32_0, Relpos_32_1,Relpos_32_2, Relpos_32_3, and Relpos_32_4 are set to 0, 0.15, 0.3, 0.7,and 0.85, respectively. Since the value of the parameter Relpos_32_0 ofthe first phase is 0, like in the case of “strong”, the preceding one ofthe two original frames in the search range memories 451 and 452 isoutput as it is from the interpolator 453.

On the other hand, the values 0.15 and 0.3 of the parameters Relpos_32_1and Relpos_32_2 of the second and third phases (as shown in FIG. 3( b),these are parameters for generating an interpolation frame closer to thefront original frame of the four interpolation frames between theneighboring original frames) are smaller than the values 0.2 and 0.4 inthe case of “strong”.

The values 0.7 and 0.85 of the parameters Relpos_32_3 and Relpos_32_4 ofthe fourth and fifth phases (as shown in FIG. 3( b), these areparameters for generating an interpolation frame closer to thesubsequent one of the four interpolation frames between the neighboringoriginal frames) are larger than the values 0.6 and 0.8 in the case of“strong”.

By the values of the parameters Relpos_32_1 to Relpos_32_4, in the caseof “medium”, the interpolation positions of the video image in the fourinterpolation frames generated between the two original frames in thesearch range memories 451 and 452 are positions of 15%, 30%, 70%, and85% of the magnitude of the motion of the video image between the twooriginal frames. That is, the interpolation positions of the videoimages in the four interpolation frames are not positions (the sameinterpolation positions as those in related art) obtained by evenlydividing the magnitude of the motion of the video image between the twooriginal frames like in the case of “strong” but are positions closer tothe video images in the original frames close to the interpolationframes than the evenly divided positions.

In the case where “weak” is selected by the interpolation positionadjustment button 401, the values of Relpos_32_0, Relpos_32_1,Relpos_32_2, Relpos_32_3, and Relpos_32_4 are set to 0, 0.1, 0.2, 0.8,and 0.9, respectively. The values 0.1 and 0.2 of the parameters of thesecond and third phases (parameters for generating interpolation framescloser to the front original frame, in the four interpolation framesbetween the neighboring original frames) are further smaller than thevalues 0.15 and 0.3 in the case of “medium”.

Further, the values 0.8 and 0.9 of the parameters of the fourth andfifth phases (parameters for generating interpolation frames closer tothe rear original frame, in the four interpolation frames between theneighboring original frames) are larger than the values 0.7 and 0.85 inthe case of “medium”.

By the values of the parameters Relpos_32_1 to Relpos_32_4, in the caseof “weak”, the interpolation positions of the video image in the fourinterpolation frames generated between the two original frames in thesearch range memories 451 and 452 are positions of 10%, 20%, 80%, and90% of the magnitude of the motion of the video image between the twooriginal frames. That is, the interpolation positions of the videoimages in the four interpolation frames are positions nearer to thevideo image in the original frame closer to the interpolation frames ascompared with the case of “medium”.

FIG. 9 is a diagram showing the values of the interpolation positionparameters Relpos set by the CPU 46 in accordance with the selectionresult of the interpolation position adjustment button 401 in the casewhere the 2:2 pulldown film signal is supplied. In the case where“strong” is selected by the interpolation position adjustment button401, the values of Relpos_22_0, Relpos_22_1, Relpos_22_2, andRelpos_22_3 are set to 0, 0.25, 0.5, and 0.75, respectively.

Since the value of the parameter Relpos_22_0 of the first phase is 0,the preceding original frame of the two original frames in the searchrange memories 451 and 452 is output as it is from the interpolator 453.

Further, since the values of the parameters Relpos_22_1, Relpos_22_2,and Relpos_22_3 of the second, third, and fourth phases change uniformlyby 0.25 like 0.25, 0.5, and 0.75, from the formulae (1) and (2), theinterpolation positions of video images in the three interpolationframes generated between the two original frames in the search rangememories 451 and 452 are the same as the interpolation positions inrelated art shown in FIG. 2( b). The positions are obtained by uniformlydividing the magnitude of a motion of a video image between the twooriginal frames into four portions, that is, positions of 25%, 50%, and75% of the magnitude of the motion.

In the case where “medium” is selected by the interpolation positionadjustment button 401, the values of Relpos_22_0, Relpos_22_1,Relpos_22_2, and Relpos_22_3 are set to 0, 0.15, 0.3, and 0.85,respectively. Since the value of the parameter Relpos_22_0 of the firstphase is 0, like in the case of “strong”, the preceding one of the twooriginal frames in the search range memories 451 and 452 is output as itis from the interpolator 453.

On the other hand, the value 0.15 of the parameter Relpos_22_1 of thesecond phase (as shown in FIG. 2( b), this is a parameter for generatingan interpolation frame closer to the front original frame in the threeinterpolation frames between the neighboring original frames) smallerthan the value 0.25 in the case of “strong”.

Further, as shown in FIG. 2( b), the parameter Relpos_22_2 of the thirdphase is a parameter for generating an interpolation frame right in themiddle of the front original frame and the rear original frame in thethree interpolation frames between the neighboring original frames.Herein, by classifying the middle interpolation frame as aninterpolation frame nearer to the front original frame, the value of theparameter Relpos_22_2 becomes the value 0.3 smaller than the value 0.5in the case of “strong”.

The value 0.85 of the parameter Relpos_22_3 of the fourth phase (asshown in FIG. 2( b), this is a parameter for generating an interpolationframe nearer to the rear original frame in the three interpolationframes between the neighboring original frames) is larger than the value0.75 in the case of “strong”.

By the values of the parameters Relpos_22_1 to Relpos_22_3, in the caseof “medium”, the interpolation positions of the video image in the threeinterpolation frames generated between the two original frames in thesearch range memories 451 and 452 are positions of 15%, 30%, and 85% ofthe magnitude of the motion of the video image between the two originalframes. That is, the interpolation positions of the video images in thethree interpolation frames are not positions (the same interpolationpositions as those in related art) obtained by evenly dividing themagnitude of the motion of the video image between the two originalframes like in the case of “strong” but are positions closer to thevideo images in the original frame nearer to the interpolation framesthan the evenly divided positions.

In the case where “weak” is selected by the interpolation positionadjustment button 401, the values of Relpos_22_0, Relpos_22_1,Relpos_22_2, and Relpos_22_3 are set to 0, 0.1, 0.2, and 0.9,respectively. The values 0.1 and 0.2 of the parameters of the second andthird phases (parameters for generating interpolation frames nearer tothe front original frame, in the three interpolation frames between theneighboring original frames) are further smaller than the values 0.15and 0.3 in the case of “medium”.

Further, the value 0.9 of the parameter of the fourth phase (parameterfor generating an interpolation frame nearer to the rear original frame,in the three interpolation frames between the neighboring originalframes) is larger than the value 0.85 in the case of “medium”.

By the values of the parameters Relpos_22_1 to Relpos_22_3, in the caseof “weak”, the interpolation positions of the video image in the threeinterpolation frames generated between the two original frames in thesearch range memories 451 and 452 are positions of 10%, 20%, and 90% ofthe magnitude of the motion of the video image between the two originalframes. That is, the interpolation positions of the video images in thethree interpolation frames are positions nearer to the video image inthe original frame closer to the interpolation frames as compared withthe case of “medium”.

FIG. 10 are diagrams showing, using the video images of the sameoriginal frame as that of FIG. 4 as an example, video images (FIG. 10(b)) subjected to the frame rate conversion in the case where the 3:2pulldown film signal is supplied to the video signal processor 4 of FIG.5 and “weak” is selected by the interpolation position adjustment button401 in comparison with video images (FIG. 10( a)) in interpolationpositions in related art.

As shown in FIG. 10( b), among four interpolation frames, in twointerpolation frames closer to the front original frame A, the image ofan airplane is positioned nearer to the original frame A as comparedwith that in the case of the related art. On the other hand, in twointerpolation frames closer to the rear original frame B, the image ofthe airplane is positioned nearer to the original frame B as comparedwith that in the case of the related art. Therefore, the intervalbetween the positions of the images of the airplane of the second andthird interpolation frames is larger than that of the related art.

As described above, in the video signal processor 4, when “weak” or“medium” is selected by the interpolation position adjustment button401, the interpolation position in an interpolation frame nearer to thefront original frame in the earlier original frame and the followingoriginal frame is shifted toward the video image of the front originalframe. The interpolation position in an interpolation frame nearer tothe rear original frame is shifted toward the video image of the rearoriginal frame.

Consequently, as also shown in FIG. 10, between the interpolation framesnearer to the front original frame and the interpolation frames nearerto the rear original frame, the interval of positions of video imagesinterpolated is larger than that in the case of the related art.

As described above, the interpolation frames whose interpolationpositions of the video images are apart from each other more than thecase of the related art. Consequently, unsmoothness of the motion of thevideo image between the interpolation frames is more conspicuous thanthe case of the related art. Therefore, at the time of performing theframe rate conversion of the film signal, while reducing a judder by theframe rate conversion, the degree of reduction is able to be loweredmore than the case of the related art.

In addition, in the case where the user watches a video image of a filmsignal by the television receiver, some users prefer the case where thejudder is reduced largely to make the motion of the video imagesmoother, and some users prefer the case where judder remains to someextent image preserves the taste of a film signal. Consequently, a userwho prefers smoother motion of a video image selects “strong” with theinterpolation position adjustment button 401. A user who prefers animage where judder remains to some extent selects “weak” or “medium”with the interpolation position adjustment button 401. Thus, the degreeof reduction of judder can be selected according to the preference ofeach user.

As described above in “Background Art”, in the frame rate conversionusing motion compensation, in the case where motion of a video imagebetween neighboring original frames becomes very quick, the motionvector lies out of the motion vector search range, so that a largejudder occurs. In such a case as well, by decreasing the degree ofreducing a judder by selecting “weak” or “medium” with the interpolationposition adjustment button 401, since a large judder occurs while theuser watches a video image including some judders, the user feels lessstrange than the conventional case where a large judder suddenly occursin a video image in which the motion is very smooth.

In addition, in the frame rate conversion using motion compensation, itis known that, as a side effect, a phenomenon (called Halo) occurs suchthat noise like shimmer is seen in the contour of a video image of amoving human or the like. The Halo becomes more conspicuous as theposition of the video image to be interpolated is apart from theposition of the video image in the original frame. On the contrary, when“weak” or “medium” is selected with the interpolation positionadjustment button 401, the position of a video image to be interpolatedbecomes nearer to the video image in the original frame, so that theHalo is able to be suppressed.

In the example of FIG. 5, the remote controller 400 is provided with theinterpolation position adjustment button 401 for switching and selectingthe value of the interpolation position parameter Relpos in three levelsof “strong, medium, and weak”. However, as another example, operatingmeans such as a volume switch for selecting the value of theinterpolation position parameter Relpos while continuously (steplessly)changing the value within the range of “strong” to “weak” in FIGS. 8 and9 may be provided for the remote controller 400 or the televisionreceiver body. In this case, to further decrease the minimum changeamount of the value of the interpolation position parameter Relpos, thenumber of bits of the interpolation position parameter Relpos suppliedfrom the CPU 46 may be set to be larger than six bits (for example,about eight bits).

Next, FIG. 11 is a block diagram showing an example of the circuitconfiguration of a video signal processor (video signal processor 4A)according to a modified example of the embodiment. The same referencenumerals are designated to the same components as those of the videosignal processor 4 shown in FIG. 5, and their description will not berepeated.

In the video signal processor 4A, the S/N level of the digital componentsignal YUV supplied to the video signal processor 4A is detected by anS/N level detector 49. Then, a signal indicative of the detection resultis sent to the CPU 46 via the I²C bus 40.

In the frame rate conversion using the motion compensation, as describedabove, the phenomenon (Halo) occurs such that noise like shimmer appearsin the contour of a video image of a moving human or the like. The Halobecomes more conspicuous as an interpolation position of a video imageis apart from the position of the video image in the original frame. Inaddition, the Halo occurs more as the S/N level of a video signaldecreases (the noise level increases).

In a memory in the CPU 46, information indicative of an S/N level of apredetermined value which is preset as a border of whether a Halo occurseasily or not is pre-stored. In the case where the detection result ofthe S/N level detector 49 is higher than the predetermined level, theCPU 46 sets the interpolation position parameter Relpos supplied to theinterpolation section 45 to the value of “strong” in FIGS. 8 and 9. Onthe other hand, when the detection result of the S/N level detector 49is equal to or less than the predetermined level, the CPU 46 sets theinterpolation position parameter Relpos supplied to the interpolationsection 45 to the value of “weak” (or “medium”) in FIGS. 8 and 9.

Thereby, in the case where the S/N level of the digital component signalYUV to be supplied is high (in the case where Halo does not easilyoccur), the motion of a video image is able to be made smooth. In thecase where the S/N level is low (in the case where Halo easily occurs),by setting the interpolation position of the video image nearer to thevideo image of the original frame, the Halo is able to be suppressed.

In addition, the case of frame-rate converting the film signal isexemplified in the embodiment. However, for example, in the case ofconverting the frame rate of a camera signal of the NTSC system to 240Hz as shown in FIG. 12, three interpolation frames are added at 1/240sec intervals between neighboring original frames (between the frames Aand B, between the frames B and C, and between the frames C and D).Although not shown, in the case of converting the frame rate of a camerasignal in the PAL system to 200 Hz, three interpolation frames are addedat 1/200 sec intervals between neighboring original frames. The presentinvention may be applied also to the case of converting a camera signalto a high frame rate as described above.

In the embodiment, the example of setting the value of the interpolationposition parameter Relpos by selecting operation of the user and theexample of setting the value of the interpolation position parameterRelpos in accordance with the S/N level of a video signal have beendescribed. However, as further another method of setting the value ofthe interpolation position parameter Relpos, for example, information ofgenres of television broadcast programs received at present is obtainedfrom an EPG (Electronic Program Guide) and the value of theinterpolation position parameter Relpos may be set according to thegenre (for example, the value of “strong” in FIGS. 8 and 9 is set in agenre in which motion of a video image is slow, and the value of “weak”or “medium” in FIGS. 8 and 9 is set in a genre in which motion of avideo image is fast).

Alternatively, the value of the interpolation position parameter Relposmay be set to the value of “weak” or “medium” in FIGS. 8 and 9 by factreset.

Further, the values of “weak” and “medium” shown in FIGS. 8 and 9 arejust an example. Obviously, by another value, the interpolation positionof a video image in each interpolation frame may be set to a positionnearer to the video image of the original frame closer to theinterpolation frame.

Furthermore, in the embodiment, the example of applying the presentinvention to the video signal processor in the television receiver hasbeen described. However, in addition to this, the invention is alsoapplicable to any video signal processor for converting the frame rateof a video signal by using motion compensation such as a video signalprocessor in a DVD player.

Second Embodiment

A second embodiment of the present invention will now be described.

FIG. 13 shows an example of the configuration of a video signalprocessor (video signal processor 4B) of the second embodiment. The samereference numerals are designated to the same components as those of theforegoing embodiment, and their description will not be repeated.

The video signal processor 4B executes various image processes on movingimage data on an access unit basis. The access unit is a unit of amoving image such as a frame or a field and, concretely, refers to, forexample, an entire picture or a part of a picture constituting a movingimage. In this case, the picture denotes here a single stationary image.Therefore, the entire picture corresponds to a frame. However,hereinafter, for simplicity of explanation, it is assumed that the videosignal processor 4B executes various image processes on moving imagedata on the frame unit basis.

As shown in FIG. 13, the video signal processor 4B is obtained byfurther providing the video signal processor 4A (including theinterpolation section 45 (high frame rate converting unit)) described inthe first embodiment with an imaging blur characteristic detector 12 andan imaging blur suppression processor 13.

To the interpolation section 45, as described in the first embodiment,for example, a moving image signal such as a television broad signal isinput as moving image data in the frame unit.

In the following, in the case where the moving image and moving imagedata corresponding to the moving image do not have to be discriminatedfrom each other, the moving image and moving image data corresponding tothe moving image will be simply called a moving image collectively.Similarly, in the case where a frame and frame data corresponding to theframe do not have to be discriminated from each other, they will besimply called a frame.

In the case where a moving image at a first frame rate is input, theinterpolation section 45 performs high frame rate converting process onthe moving image and supplies a moving image of a second frame ratehigher than the first frame rate, obtained as a result of the process,to the imaging blur characteristic detector 12 and the imaging blursuppression processor 13.

The high frame rate converting process is a process executed in the casewhere the first frame rate at the time of input is lower than the secondframe rate at the time of output (display). It is a process ofconverting the first frame rate to the second frame rate higher than thefirst frame rate by creating a new frame and inserting it between eachof frames constructing a moving image at the time of input.

In this case, the first frame rate refers to a frame rate of a movingimage at the time point when the moving image is input to theinterpolation section 45. Therefore, the first frame rate can be anarbitrary frame rate. In this case, for example, it is a frame rate whena moving image is captured by a not-shown imaging apparatus, that is, animaging frame rate.

Further, in the embodiment, as an example of the high frame rateconverter for performing such high frame rate converting process, theinterpolation section 45 described in the first embodiment will bedescribed (in the case of adding N pieces of interpolation framesbetween neighboring original frames, as the interpolation positions ofthe video image in the interpolation frames, not positions obtained byequally dividing the magnitude of the motion of the video image betweenthe earlier original frame and the following original frame, positionscloser to the video images of original frames nearer to theinterpolation frames than the evenly divided positions are set). Inplace of the interpolation section 45, a normal high frame rateconverter (which sets, as video image interpolation positions ininterpolation frames, positions obtained by evenly dividing themagnitude of the motion of the video image between the earlier originalframe and the following original frame) may be provided.

The imaging blur characteristic detector 12 detects the value of aparameter indicative of the characteristic of an imaging blur withrespect to each of the frames constructing a moving image supplied fromthe interpolation section 45. The detection result of the imaging blurcharacteristic detector 12, that is, the value of the parameterindicative of the characteristic of the imaging blur is supplied to theimaging blur suppression processor 13.

In addition, the parameter indicative of the characteristic of theimaging blur is not limited but various parameters are able to beemployed. Concrete examples of the parameter indicative of thecharacteristic of such an imaging blur will be described later. Forexample, in the case of using the absolute value of a travel vector(motion vector) as the parameter indicative of the characteristic of theimaging blur, the imaging blur characteristic detector 12 may includethe motion vector detector 44 described in the first embodiment.

The number of detected values of the parameter indicative of thecharacteristic of the imaging blur in one frame is not particularlylimited. For example, only one value of the parameter indicative of thecharacteristic of an imaging blur may be detected per frame. The valueof the parameter indicative of the characteristic of the imaging blurmay be detected for each of the pixels constructing the frame. It isalso possible to divide the one frame into some blocks and detect thevalue of the parameter indicative of the characteristic of the imagingblur for each of the divided blocks.

The imaging blur suppression processor 13 corrects the value of each ofpixels constructing a frame to be processed on the basis of a valuecorresponding to the frame to be processed in the values of theparameter detected by the imaging blur characteristic detector 12 withrespect to each of the frames constructing a moving image supplied fromthe interpolation section 45. Namely, according to the characteristic(the value of the parameter) of the imaging blur in the frame to beprocessed, the imaging blur suppression processor 13 corrects each ofthe pixel values of the frame to be processed so as to suppress theimaging blur. That is, using the detected value of the parameter, theimaging blur suppressing process of suppressing deterioration in picturequality caused by the imaging blur included in each of the framessupplied from the interpolation section 45 is performed.

Thereby, a moving image in which the imaging blur is suppressed bycorrecting each of the pixel values of each of the frames and which isconverted to the second frame rate higher than the first frame rate atthe time of input is output from the imaging blur suppression processor13 to the outside of the video signal processor 4B.

In the example of FIG. 13, the set of the imaging blur characteristicdetector 12 and the imaging blur suppression processor 13 is used incombination with the interpolation section 45. However, naturally, theset may be used by itself, or can be used in combination with anot-shown another function block (another video image signal processorfor performing a predetermined image process).

That is, only by the set of the imaging blur characteristic detector 12and the imaging blur suppression processor 13, the effect of suppressingthe imaging blur can be produced. However, to make the effect moreconspicuous, it is preferable to combine the set of the imaging blurcharacteristic detector 12 and the imaging blur suppression processor 13with the interpolation section 45 as described above. The reason will bedescribed below.

A blur recognized by a human when a moving image displayed on anot-shown display apparatus is formed as an image on the retina of thehuman is a combination of a hold blur which occurs when the humanfollows and sees a moving object included in the moving image and theabove-mentioned imaging blur added at the time of capture of the movingimage.

The characteristic of the imaging blur is expressed as a low-pass filteras will be described later with reference to FIG. 16 and the like.Specifically, an image signal after the imaging blur is a signalequivalent to a signal obtained by passing an image signal without animaging blur (ideal image signal) through the low-pass filter.Therefore, the frequency characteristic of the image signal with theimaging blur is worse than that of the image signal without the imagingblur. That is, generally, the higher the frequency is, the lower thegain of the image signal with the imaging blur is as compared with thatof the image signal without the imaging blur.

The characteristic of the hold blur is also expressed as the low-passfilter like the characteristic of the imaging blur. That is, an imagesignal with the hold blur is a signal equivalent to a signal obtained bypassing an image signal without a hold blur (the image signal with theimaging blur) through the low-pass filter. Therefore, the frequencycharacteristic of the image signal with the hold blur is worse than thatof the image signal without the hold blur. That is, generally, thehigher the frequency is, the lower the gain of the image signal with thehold blur is as compared with that of the image signal without the holdblur. However, the hold blur occurs only in the case where the displayapparatus is a fixed-pixel (hold) display apparatus.

Therefore, by performing the high frame rate converting process on theimage signal with the imaging blur whose frequency characteristic hasalready deteriorated due to the imaging blur, the hold blur is able tobe suppressed. However, even if the high frame rate converting processis performed, the deterioration of the imaging blur is unchanged and,finally, the effect of suppressing the blur on the retina of a human ishalved. This will be described with reference to FIG. 14.

FIG. 14 shows a frequency characteristic of a blur in an image formed onthe retina of a human in the case where an image of a real object movingat a travel speed 4 [pixel/frame] is captured in an image capture rangeof an image capturing apparatus (hereinbelow, called a camera). In FIG.14, the horizontal axis denotes frequency, and the vertical axisindicates gain. However, each value on the horizontal axis denotesrelative values in the case where the Nyquist frequency is 1.

In FIG. 14, a curve h0 indicated by an alternate long and short dashline shows the frequency characteristic of a blur in an image formed onthe retina of a human, in the case where a process for reducing blurs(including the imaging blur and the hold blur) is not performed.Specifically, in the case where a moving image, which is input to thevideo signal processor 4B in the example of FIG. 13, is directlysupplied to the display apparatus and displayed without being input tothe video signal processor 4B (without being processed), the frequencycharacteristic of a blur in an image formed on the retina of a human whosees the moving image is the curve h0.

In contrast, for example, when the display speed is doubled by the highframe rate converting process, only the hold blur is reduced. As aresult, the frequency characteristic of a blur in an image formed on theretina of a human becomes a curve h1 shown by a dotted line in thediagram. Specifically, in the case where a moving image which is inputto the video signal processor 4B in FIG. 13 is subjected to the highframe rate converting process in the interpolation section 45 and thenis supplied to the display apparatus and displayed without being inputto the imaging blur suppression processor 13 (without reducing theimaging blur), the frequency characteristic of a blur in an image formedon the retina of a human who sees the moving image is the curve h1.

For example, when the display speed is doubled by the high frame rateconverting process (the hold blur is reduced), and the degree of theimaging blur is reduced by half by applying the present invention, thefrequency characteristic of a blur in an image formed on the retina of ahuman becomes a curve h2 indicated by the solid line in the diagram.Specifically, in the case where a moving image which is input to thevideo signal processor 4B in FIG. 13 is subjected to the high frame rateconverting process in the interpolation section 45, subject to imagingblur suppression by the imaging blur suppression processor 13, and thenis supplied to the display apparatus and displayed, the frequencycharacteristic of a blur in an image formed on the retina of a human whosees the moving image is the curve h2.

It is understood from comparison between the curves h1 and h2 thatreduction only in the hold blur by the high frame rate convertingprocess is insufficient for reduction in the characteristic of the bluron the retina of a human, and further reduction in the imaging blur isnecessary. However, as described above, in the technique of the relatedart, the high frame rate converting process is simply performed withoutparticularly considering necessity of reduction in the imaging blur.

Therefore, the video signal processors of the present invention, in theembodiment of FIG. 13 and embodiments of FIGS. 35 and 36 and the likewhich will be described later is provided with not only theinterpolation section 45 but also the imaging blur characteristicdetector 12 and the imaging blur suppression processor 13 in order toreduce the imaging blur, that is, to improve the characteristic of theblur on the retina of a human from the curve h0 to curve h2 in FIG. 14.However, as described in the embodiment of FIGS. 37 and 38, the imagingblur characteristic detector 12 is not an essential component for thevideo signal processor of the present invention.

That is, the imaging blur suppression processor 13 corrects each of thepixel values of each of frames to be processed on the basis of the valuecorresponding to the frames to be processed in the values of parametersindicative of characteristics of the imaging blur detected by theimaging blur characteristic detector 12, thereby suppressingdeterioration in the image caused by the imaging blur in the framessubjected to the high frame rate conversion. In other words, bysupplying an image signal output from the video signal processor of thepresent invention such as the video signal processor 4B to the not-showndisplay apparatus, the display apparatus is able to display a clearimage in which the image deterioration (blur image) is suppressed as animage corresponding to the image signal.

As described above, it is preferable to combine the set of the imagingblur characteristic detector 12 and the imaging blur suppressionprocessor 13 with the interpolation section 45.

Next, with reference to the flowchart of FIG. 15, the image process ofthe video signal processor 4B having the functional configuration ofFIG. 13 will be described.

In step S1, the interpolation section 45 inputs a moving image of thefirst frame rate.

In step S2, the interpolation section 45 converts the frame rate of themoving image to the second frame rate higher than the first frame rate.

When the moving image converted from the first frame rate to the secondframe rate is supplied from the interpolation section 45 to the imagingblur detector 12 and the imaging blur suppression processor 13, theprocess advances to step S3.

In step S3, the imaging blur characteristic detector 12 detects one ormore values of parameters indicative of the characteristics of theimaging blur in each of the frames constructing the moving image.

When the one or more values of parameters indicative of thecharacteristics of the imaging blur in each of the frames constructingthe moving image are supplied from the imaging blur characteristicdetector 12 to the imaging blur suppression processor 13, the processadvances to step S4.

In step S4, with respect to each of the frames constructing the movingimage supplied from the interpolation section 45, the imaging blursuppression processor 13 corrects each of the pixel values of the frameto be processed on the basis of one or more values corresponding to theframe to be processed among the values of the parameter detected by theimaging blur detector 12.

In step S5, the imaging blur suppression processor 13 outputs the movingimage obtained by correcting the pixel values of each of the frames andchanging the first frame rate to the second frame rate.

After that, the image process of FIG. 15 is finished.

In the above description, for simplicity of the explanation, theprocesses in each of steps S1 to S5 are performed on the moving imageunit basis. In reality, however, the frame is often the process unit.

In the image process of FIG. 15, the fact that the process unit of eachstep is a moving image is equivalent to the fact that the condition ofmoving the step to be processed in the steps S1 to S5 to the next stepis a condition that the process of the step to be processed is performedon an entire moving image.

On the other hand, in the image process of FIG. 15, the fact that theprocess unit in each of the steps is a frame is equivalent to the factthat the condition of moving the step to be processed in the steps S1 toS5 to the next step is a condition of performing the process of the stepto be processed on an entire frame. In other words, the state where theprocess unit in each of the steps is a frame is equivalent to the statewhere the continuous processes in the steps S1 to S5 on each of theframes are executed independently from (in parallel with) another frame.In this case, for example, when the process in the step S3 is executedon the first frame, the process in the step S2 on the second framedifferent from the above may be executed in parallel.

Further, in reality, it often happens that each of pixels constructing aframe to be processed is sequentially set as a pixel to be noted as anobject of the process (hereinbelow, called a target pixel) and, atleast, the processes in the steps S3 and S4 are sequentially andindividually performed on the target pixel. That is, the process unit inthe steps S3 and S4 is often a pixel.

In the following description, therefore, it will be also assumed thatthe processes in the steps S3 and S4 are performed on the pixel unitbasis. Specifically, the process in the step S3 is a process of theimaging blur characteristic detector 12. The process in the step S4 is aprocess of the imaging blur suppression processor 13. Therefore, thefollowing description will be given on assumption that the process unitof the imaging blur characteristic detector 12 and the imaging blursuppression processor 13 is a pixel.

Next, the details of the imaging blur suppression processor 13 in thevideo signal processor 4B in FIG. 13 will now be described. Concretely,for example, an embodiment of the imaging blur suppression processor 13in the case of using the absolute value of a travel vector (motionvector) as a parameter indicative of the characteristic of the imagingblur will be described.

In the following, the absolute value of the travel vector (motionvector) will be called travel speed, and the direction of the travelvector (motion vector) will be called a travel direction. The traveldirection can be any direction on a two-dimensional plane. Naturally, inthe case where any direction on a two-dimensional plane becomes thetravel direction, the video signal processor 4B of FIG. 13 can similarlyexecute various processes which will be described later. However, in thefollowing, for simplicity of explanation, it is assumed that the traveldirection is the lateral direction.

In the case where the travel speed is used as a parameter indicative ofthe characteristic of the imaging blur, for example, for each of theframes constructing a moving image, the imaging blur characteristicdetector 12 sequentially sets each of the pixels constructing the frameto be processed as a target pixel, sequentially detects a travel vectorin the target pixel, and sequentially supplies the travel vector as avalue of the parameter indicative of the characteristic of the imagingblur in the target pixel to the imaging blur suppression processor 13.

Therefore, for example, for each of the frames constructing a movingimage, the imaging blur suppression processor 13 sequentially sets eachof the pixels constructing the frame to be processed as a target pixel,and sequentially corrects the pixel value of the target pixel on thebasis of the travel speed in the target pixel supplied from the imagingblur characteristic detector 12.

Here, the reason why the travel speed can be employed as a parameterindicative of the characteristic of the imaging blur will be described.

The characteristic of the imaging blur can be generally expressed in theform that it depends on the travel speed of a subject.

In addition, in the case where a subject itself moves in a real spaceand a camera is fixed, the travel speed of the subject naturallyincludes the travel speed of a subject (image) in a frame when thesubject is captured by the camera. Further, the travel speed of thesubject here includes travel speed relative to the subject (image) inthe frame when the subject is captured by the camera in the case wherethe subject is fixed in the real space and the camera is moved by a handshake or the like or in the case where both the subject and the cameramove in the real space.

Therefore, the characteristic of the imaging blur can be expressed inthe form that it depends on the travel speed in each of pixelsconstructing an image of a subject.

The travel speed in a pixel refers to a spatial distance between a pixelin a frame to be processed and a corresponding pixel (correspondencepoint) in the preceding frame. For example, in the case where a spatialdistance between a pixel in a frame to be processed and a correspondingpixel (correspondence point) in the immediately preceding frame(temporally one before) is v pixels (v denotes an arbitrary integervalue equal to or larger than 0), the travel speed in the pixel is v[pixels/frame].

In this case, if predetermined one of each pixel constructing an imageof a subject is set as a target pixel, the characteristic of the imagingblur in the target pixel can be expressed in the form that depends onthe travel speed v [pixels/frame] in the target pixel.

More concretely, for example, in the case where the travel speed of thetarget pixel is 2, 3, and 4 [pixels/frame], the frequencycharacteristics of the imaging blur in the target pixel can be expressedby curves H2, H3, and H4, respectively, in FIG. 16.

That is, FIG. 16 shows the frequency characteristics of the imaging blurin the target pixel in the case where the travel speed in the targetpixel is 2, 3, and 4 [pixels/frame]. In FIG. 16, the horizontal axisshows frequency, and the vertical axis shows gain. However, each valueon the horizontal axis shows relative values in the case where theNyquist frequency is 1.

The reason why the travel speed can be employed as a parameterindicative of the characteristic of the imaging blur has been describedabove.

By the way, as understood from the frequency characteristics H2 to H4 inFIG. 16, when the characteristic of an imaging blur in a target pixel isexpressed in a space region, it can be expressed by a moving averagefilter (low-pass filter).

Specifically, when a transfer function indicative of the moving averagefilter (low-pass filter) (hereinbelow, called transfer function of theimaging blur) is written as H, an ideal image signal in the case whereno imaging blur supposedly occurs (hereinbelow, called a signal withoutan imaging blur) is expressed as F in a frequency area, and an actualimage signal output from a camera, that is, an image signal in which animaging blur occurs (hereinbelow, called a signal with an imaging blur)is expressed as H in the frequency area, a signal G with the imagingblur is expressed as the following equation (3).

G=H×F  (3)

An object of the invention is to remove (suppress) the imaging blur. Toachieve the object, it is sufficient to perform forecasting calculationthe signal F without the imaging blur from the signal G with the imagingblur which is known and the transfer function H of the imaging blurwhich is known. That is, it is sufficient to execute the followingequation (4) of forecasting calculation.

F=inv(H)×G  (4)

In the equation (4), inv(H) indicates inverse function of the transferfunction H of the imaging blur. Since the transfer function H of theimaging blur has the characteristic of a low-pass filter as describedabove, the inverse function inv(H) of the transfer function H naturallyhas the characteristic of a high-pass filter.

As described above, the characteristic of the transfer function H of theimaging blur varies according to the travel speed. Concretely, forexample, when the travel speed in the target pixel is 2, 3, and 4[pixels/frame], the frequency characteristic of the transfer function Hof the imaging blur in the target pixel becomes differentcharacteristics as shown by curves H2, H3, and H4, respectively, in FIG.16.

Thereby, the imaging blur suppression processor 13 can achieve theobject of the present invention, that is, the object of removing(suppressing) the imaging blur by changing the characteristic of thetransfer function H of the imaging blur in accordance with the travelspeed, obtaining the inverse function inv(H) of the transfer function Hwhose characteristic was changed, and executing the computing process ofthe above-mentioned equation (4) using the inverse function inv(H).

Alternately, since the computation of the above-mentioned equation (4)is computation of a frequency region, to achieve the object of thepresent invention, the imaging blur suppression processor 13 may executea process in a space region equivalent to the computing process of theabove-mentioned equation (4). Concretely, for example, the imaging blursuppression processor 13 may execute the following first to thirdprocesses.

In the first process, according to the travel speed in a target pixelsupplied from the imaging blur characteristic detector 12, thecharacteristic of the moving average filter (low-pass filer) expressingthe imaging blur in the target pixel is converted. Concretely, forexample, moving average filters are prepared for plural travel speeds ina one-to-one corresponding manner. A process of selecting one filtercorresponding to the travel speed in the target pixel among a pluralityof moving average filters is an example of the first process.

The second process is a process made of the following processes 2-1 to2-3.

The process 2-1 is a process of displaying the moving average filter infrequency by performing Fourier transform on the moving average filterwhose characteristic is converted by the first process. Concretely, forexample, in the case where the travel speed in the target pixel is 2, 3,and 4 [pixels/frame], the process of obtaining the curves H2, H3, and H4in FIG. 16 is the process 2-1. Namely, from the viewpoint of thefrequency region, the process of obtaining the transfer function H ofthe imaging blur in the target pixel is the process 2-1.

The process 2-2 is a process of calculating the inverse of the movingaverage filter which is frequency indicated by the process 2-1. That is,from the viewpoint of the frequency region, the process of generatingthe inverse function inv(H) of the transfer function H of the imagingblur expressed by the above-mentioned equation (4) is the process 2-2.

The process 2-3 is a process of performing the inverse Fourier transformon the inverse of the moving average filter which is calculated by theprocess 2-2 and is frequency indicated. That is, a process of generatinga high-pass filter (Wiener filter or the like) corresponding to theinverse function inv(H) is the process 2-3. In other words, the processof generating an inverse filter of the moving average filter is theprocess 2-3. In the following, the high-pass filter generated by theprocess 2-3 will be called an inverse moving average filter.

The third process is a process of inputting, as an input image, an imagesignal g in the space region corresponding to the signal G in theabove-mentioned equation (4) in the frequency range with the imagingblur, and applying the inverse moving average filter generated by theprocess 2-3 on the image signal g. By the third process, an image signalf in the space region corresponding to the signal F in theabove-mentioned equation (4) in the frequency region without the imagingblur is reconstructed (forecasting-calculated). Concretely, for example,a process of correcting the pixel value of the target pixel by applyingthe inverse moving average filter on a predetermined block including thetarget pixel in the frame to be processed is the third process.

An embodiment of the functional configuration of the imaging blursuppression processor 13 capable of executing the first to thirdprocesses has been already invented by the inventors of the presentinvention and is disclosed in FIG. 17 submitted together with theapplication of Japanese Patent Application No. 2004-234051.

However, in the case where the imaging blur suppression processor 13 hasthe configuration of FIG. 17 presented together with the application ofJapanese Patent Application No. 2004-234051, a first issue as describedbelow newly occurs. That is, as also shown by the frequencycharacteristics H2 to H4 in FIG. 16, the moving average filter (itsfrequency characteristic) indicative of the imaging blur includes thefrequency at which the gain becomes zero. Consequently, it is difficultfor the imaging blur suppression processor 13 to generate a completeinverse filter of the moving average filter (complete inverse movingaverage filter). As a result, the first issue that noise increases newlyoccurs.

Further, the process of applying the high-pass filter (inverse movingaverage filter) on the image signal like the third process can be alsosaid a process of making an edge sharp. As an image forming technique inthe meaning of “making an edge sharp”, in the past, techniques such asLTI and sharpness exist. Obviously, such a conventional technique can beapplied to the imaging blur suppression processor 13.

However, in the case of applying such a conventional technique to theimaging blur suppression processor 13, the following second to fifthissues newly occur.

That is, the LTI is a technique of the related art disclosed in JapaneseUnexamined Patent Application Publication No. 2000-324364 and the like.In the Japanese Unexamined Patent Application Publication No.2000-324364, a technique of replacing the luminance (pixel value) of thetarget pixel with the luminance (pixel value) of a pixel neighboring thetarget pixel by a hard switch to correct the luminance of the targetpixel, thereby sharpening an edge is the LTI. Therefore, due to thecharacteristic, the LTI has a second issue such that durability againstnoise is low, and a process image may be damaged by noise. There is alsoa third issue that all of edges are sharpened regardless of image dataprior to the LTI.

In addition, since the techniques of the related art (LTI and sharpness)are used for image formation, the techniques have a fourth issue thatthe process is similarly performed also on a still picture in which noimaging blur occurs and a fifth issue that the process is uniformlyperformed irrespective of the amount of an imaging blur.

Accordingly, the inventors of the present invention have invented theimaging blur suppression processor 13 having, for example, thefunctional configuration shown in FIG. 17 of the present invention tosolve the issues described above in “Problems to be solved by theInvention” along with the first to fifth issues. That is, FIG. 17 showsan example of the functional configuration of the imaging blursuppression processor 13 to which the preset invention is applied.

In the example of FIG. 17, the imaging blur suppression processor 13 isconfigured to have a high frequency component removing unit 21, a filterunit 22, and an imaging blur compensating unit 23.

At least in the description of the imaging blur suppression processor13, signals input to each functional block (including computing unitssuch as an adder) constructing the imaging blur suppression processor 13is hereinafter accordingly referred to as input signals collectivelyirrespective of an input unit such as a moving image, each of framesconstructing a moving image, and a pixel value of each of pixelsconstructing each frame. Similarly, signals output from each functionalblock is hereinafter accordingly referred to as output signalscollectively regardless of an output unit. In other words, in the casewhere an input unit and an output unit have to be discriminated fromeach other, description will be given using the unit (mainly, the pixelvalue). In the other case, description will be given simply using aninput signal or an output signal.

As shown in FIG. 17, an output signal of the interpolation section 45 issupplied to the high frequency component removing unit 21 as an inputsignal to the imaging blur suppression processor 13. An output signal ofthe imaging blur characteristic detector 12 is supplied to the filterunit 22 and the imaging blur compensating unit 23. An output signal ofthe high frequency component removing unit 21 is supplied to the filterunit 22. An output signal of the filter unit 22 is supplied to theimaging blur compensating unit 23. An output signal of the imaging blurcompensating unit 23 is output to the outside as an output signalindicative of the final process result of the imaging blur suppressionprocessor 13.

In the following, the details of the high frequency component removingunit 21, the filter unit 22, and the imaging blur compensating unit 23will be described in that order.

First, with reference to FIGS. 18 and 19, the details of the highfrequency component removing unit 21 will be described.

FIG. 18 shows an example of a detailed functional configuration of thehigh frequency component removing unit 21. FIG. 19 shows thecharacteristic of an after-mentioned high-frequency limiter 32 in thehigh frequency component removing unit 21 in FIG. 18.

In the example of FIG. 18, the high frequency component removing unit 21is configured to have a high-pass filter 31, the high-frequency limiter32, and a subtractor 33.

As shown in FIG. 18, an output signal of the interpolation section 45 issupplied as an input signal to the high frequency component removingunit 21 to the high-pass filter 31 and the subtractor 33.

The high-pass filter 31 has the function of an HPF (High-Pass Filter).Therefore, the high-pass filter 31 extracts a high frequency componentfrom an input signal of the high frequency component removing unit 21and supplies it to the high-frequency limiter 32.

The high-frequency limiter 32 has a function shown by a curve P1 in FIG.19, assigns the high frequency component supplied from the high-passfilter 31 as an input parameter to the function, and supplies an outputof the function (output of FIG. 19) to the subtractor 33. That is, aseasily understood from the shape of the curve P1 in FIG. 19, thehigh-frequency limiter 32 limits the value of the high frequencycomponent (input) supplied from the high-pass filter 31 in the casewhere the value is a predetermined value or larger, or a predeterminedvalue or less. In other words, the high-frequency limiter 32 has acharacteristic shown by the curve P1 in FIG. 19.

Referring again to FIG. 18, the subtractor 33 calculates the differencebetween the input signal of the high frequency component removing unit21 and the high frequency component limited by the high-frequencylimiter 32, and supplies a derived differential signal as an outputsignal of the high frequency component removing unit 21 to the filterunit 22.

In such a manner, high frequency components such as noise are removedfrom the input signal in the high-frequency component removing unit 21,and a signal obtained as a result is supplied as an output signal to thefilter unit 22.

Next, referring to FIGS. 20 to 22, the details of the filter unit 22will be described.

FIG. 20 shows an example of a detailed functional configuration of thefilter unit 22. FIG. 21 shows an example of a detailed functionalconfiguration of a gain controller 53 which will be described later, inthe filter unit 22 in FIG. 20. FIG. 22 shows the characteristic of anadjustment amount determining unit 64 which will be described later, inthe gain controller 53 in FIG. 21.

In the example of FIG. 20, the filter unit 52 includes a moving averagefilter 51 to an adder 54.

As shown in FIG. 20, an output signal of the high frequency componentremoving unit 21 is supplied as an input signal of the filter unit 22 toeach of the moving average filter 51, a subtractor 52, and the adder 54.Further, an output signal of the imaging blur characteristic detector 12is supplied to each of the moving average filter 51 and the gaincontroller 53.

The moving average filter 51 applies moving average filtering to theinput signal of the filter unit 22. More specifically, the movingaverage filter 51 applies the moving average filter on each of pixelvalues of a predetermined block including the target pixel in a frame tobe processed in an input signal of the filter unit 22, therebycorrecting the pixel value of a target pixel. At this time, the movingaverage filter 51 converts the characteristic of the moving averagefilter in accordance with the travel speed in the target pixel in theoutput signal of the imaging blur characteristic detector 12.Concretely, for example, in the case where the travel speed in thetarget pixel is 2, 3, and 4 [pixels/frame], in view of the frequencyregion, the moving average filter 51 converts the characteristic of themoving average filter to those mentioned above shown by the curves H2,H3, and H4, respectively, in FIG. 16. The pixel value of the targetpixel corrected by the moving average filter 51 is supplied to thesubtractor 52.

At this time, the moving average filter 51 can also change the number oftaps (the target pixel and predetermined pixels neighboring the targetpixel) used in the case of writing the moving average filter on thetarget pixel in accordance with the travel speed in the target pixel inthe output signal of the imaging blur characteristic detector 12.Concretely, for example, the moving average filter 51 should vary thenumber of taps so as to be increased (that is, so as to increase widthto be averaged) as the travel speed increases. The imaging blurcompensating unit 23 uses the result of the moving average filter usingtaps of the number according to the travel speed, thereby enablingcorrection of higher precision, that is, correction capable ofsuppressing the imaging blur more to be performed.

The subtractor 52 obtains the difference between a pixel value beforecorrection of the target pixel in the input signal of the filter unit 22and the pixel value of the target pixel corrected by the moving averagefilter 51, and supplies the difference value to the gain controller 53.Hereinafter, the output signal of the subtractor 52 is called thedifference between signals before and after the moving average filter.

The gain controller 53 adjusts the value of the difference betweensignals before and after the moving average filter, and supplies, as anoutput signal, the adjusted difference between signals before and afterthe moving average filter to the adder 54. The details of the gaincontroller 53 will be described later with reference to FIG. 21.

The adder 54 adds the input signal of the filter unit 22 and the outputsignal of the gain controller 53, and supplies the addition signal as anoutput signal to the imaging blur compensating unit 23. Specifically,when attention is paid to the target pixel, the adder 54 adds, as acorrection amount, the adjusted value of the difference between thesignals before and after the moving average filter of the target pixelto the pixel value of the target pixel prior to the correction, andsupplies the addition value as the pixel value of the corrected targetpixel to the imaging blur compensating unit 23 on the outside.

The process in the space region in the filter unit 22 as described abovewill be performed as follows in view of the frequency region.

That is, in the case where the difference between the signals before andafter the moving average filter as the output signal of the subtractor52 is considered in the frequency region, when attention is paid to apredetermined frequency, the gain of the output signal of the subtractor52 becomes as follows. Specifically, at the noted frequency, thedifferential gain between the gain of the input signal of the filterunit 22 and the gain of the input signal passed through the movingaverage filter becomes the gain of the output signal of the subtractor52. The gain of the output signal of the subtractor 52 is hereinafterreferred to as the differential gain between gains before and after themoving average filter.

Further, the differential gain between gains before and after the movingaverage filter is adjusted by the gain controller 53. The gainadjustment will be described later.

Therefore, in the case where the output signal of the filter unit 22(adder 54) in the example of FIG. 20 is considered in the frequencyregion, when attention is paid to a predetermined frequency, the gain ofthe output signal is an addition gain obtained by adding the gain of theinput signal and the differential gain between gains before and afterthe moving average filter after the gain adjustment. That is, at each ofthe frequencies, the gain of the output signal is higher than that ofthe input signal only by the amount of the differential gain betweensignals before and after the moving average filter after the gainadjustment.

In other words, the filter unit 22 as a whole executes a processbasically equivalent to a process of applying a high-pass filter.

Referring to FIG. 21, the details of the gain adjuster 53 will bedescribed.

In an example of FIG. 21, the gain controller 53 has delay units 61-1 to61-n (hereinbelow, called DL units 61-1 to 61-n corresponding to FIG.21), a MAX/MIN calculator 62, a subtractor 63, the adjustment amountdetermining unit 64, and a multiplier 65.

As shown in FIG. 21, the difference between signals before and after themoving average filter as an output signal of the subtractor 52 issupplied as an input signal to the gain controller 53 to the DL unit61-1. The output signal of the imaging blur characteristic detector 12is supplied to the MAX/MIN calculator 62.

With such a configuration, the gain adjuster 53 can suppress ringingwhich occurs in a place where the level of a signal is high.

The detailed functional configuration (connection mode of eachfunctional block) of the gain controller 53 and its operation will bedescribed below.

The DL units 61-1 to 61-n are connected in that order. When an outputsignal of a preceding DL unit is supplied as an input signal to a DLunit, the DL unit delays the input signal by predetermined delay time,and supplies the resultant signal as an output signal to a subsequent DLunit. Each of output signals of the DL units 61-1 to 61-n is suppliedalso to the MAX/MIN calculator 62. Further, an output of the DL unit61-(n/2) is also supplied to the multiplexer 65.

Values corresponding to n pixels arranged successively in the traveldirection (in this case, the lateral direction) using the target pixelas a center in the difference between signals before and after themoving average filter as an input signal of the gain controller 53(hereinafter referred to as the differential values of neighboringpixels) are sequentially input to the DL unit 61-1 in the arrangementorder of the pixels from right to left. Therefore, after time n times aslong as delay time nearly elapses since then, one differential value ofneighboring pixels in the n pixels arranged successively in the lateraldirection using the target pixel as a center is output one by one fromeach of the DL units 61-1 to 61-n and supplied to the MAX/MIN calculator62. Further, the differential value of neighboring pixels of the targetvalue is output from the DL unit 61-(n/2) and is supplied to the MAX/MINcalculator 62 as described above and is also supplied to the multiplier65.

In addition, the number n of DL units 61-1 to 61-n is, though notparticularly limited, the highest value [pixels/frame] of the travelspeed in this case. It is also assumed that the travel speed in thetarget pixel supplied from the imaging blur characteristic detector 12is v [pixels/frame]. However, v is an arbitrary integer value of 0 orlarger.

The MAX/MIN calculator 62 determines, as a calculation range, a rangeincluding the target pixel as a center and including differential valuesof neighboring pixels in the v pixels of the amount corresponding to thetravel speed. The MAX/MIN calculator 62 obtains a maximum value MAX anda minimum value MIN from v differential values of neighboring pixelsincluded in the calculation range from the n differential values ofneighboring supplied from the DL units 61-1 to 61-n, and supplies themto the subtractor 63.

In addition, the range including the target pixel as a center andincluding differential values of neighboring pixel in the v pixels ofthe amount corresponding to the travel speed is set as the calculationrange for the following reason. That is, ringing exerts an influenceonly by the number of taps of the high-pass filter, in other words, onlyby the amount corresponding to the travel speed.

The subtractor 63 obtains the difference between a maximum value MAX anda minimum value MIN supplied from the MAX/MIN calculator 62 and suppliesthe differential value (=MAX−MIN) to the adjustment amount determiningunit 64.

It is known that the larger the differential value (=MAX−MIN) becomes,the larger ringing around the target pixel becomes. That is, thedifference (=MAX−MIN) is a value as the index of magnitude of theringing around the target pixel.

Then, the adjustment amount determining unit 64 determines theadjustment amount on the differential values of the pixels neighboringthe target pixel on the basis of the differential value (=MAX−MIN)supplied from the subtractor 63, and supplies it to the multiplier 65.

Specifically, for example, the adjustment amount determining unit 64holds a function indicated by the curve P2 in FIG. 22, assigns thedifferential value (=MAX−MIN) supplied from the subtractor 63 as aninput parameter to the function, and supplies an output of the function(the output of FIG. 22) as the adjustment amount on the differentialvalues of pixels neighboring the target pixel to the multiplier 65. Thatis, as easily understood from the shape of the curve P2 in FIG. 22,after the differential value (=MAX−MIN) supplied from the subtractor 63exceeds a predetermined value, the adjustment amount (output) decreasesin order to suppress occurrence of ringing. In other words, theadjustment amount determining unit 64 has the characteristic shown bythe curve P2 in FIG. 22.

Referring again to FIG. 21, the multiplier 65 multiplies thedifferential value of pixels neighboring the target pixel supplied fromthe DL unit 61-(n/2) with the adjustment amount supplied from theadjustment amount determining unit 64 (in the example of FIG. 22, thevalue in the range of 0 to 1), and supplies the resultant value as theadjusted differential value between the signals neighboring the targetpixel to the adder 54. That is, the difference values of the neighboringpixels adjusted are sequentially supplied as output signals of the gaincontroller 53 to the adder 54.

As described above, when the differential value (=MAX−MIN) as an outputsignal of the subtractor 63 exceeds a predetermined value, as thedifferential value (=MAX−MIN) increases, the adjustment amount (output)also gradually decreases toward 1 to 0. Therefore, in the case where thedifferential value (=MAX−MIN) as an output signal of the subtractor 63is equal to or larger than a predetermined value, an adjustment valuewhich is less than 1 is multiplied to the differential value of pixelsneighboring the target pixel. Thus, the difference of the pixelsneighboring the target pixel is adjusted so as to be decreased. As aresult, ringing around the target pixel is suppressed.

In view of the frequency region, it can be said, as a result, that theprocess in the space region in the gain controller 53 as described aboveis a process of adjusting the differential gain between gains before andafter the moving average filter in order to suppress ringing.

Next, referring to FIGS. 23 to 31, the details of the imaging blurcompensating unit 23 will be described.

FIG. 23 shows an example of a detailed functional configuration of theimaging blur compensating unit 23.

In the example of FIG. 23, the imaging blur compensating unit 23 isconfigured to have an ALTI unit 81, a subtractor 82, a gain controller83, and an adder 84.

As shown in FIG. 23, an output signal of the filter unit 22 is input, asan input signal of the imaging blur compensating unit 23, to the ALTIunit 81, the subtractor 82, and the adder 84. An output signal of theimaging blur characteristic detector 12 is supplied to the ALTI unit 81and the gain controller 83.

Paying attention to the pixel value of a target pixel in an input signalof the imaging blur compensating unit 23, each of the ALTI unit 81 tothe adder 84 will be hereinafter described.

As described above, the pixel value of a target pixel at the stage whenit is supplied to the imaging blur compensating unit 23 is oftendifferent from that at the stage when it is input to the imaging blursuppression processor 13 in FIG. 17 since it is already corrected by thehigh frequency component removing unit 21 and the filter unit 22.Further, as will be described later, the pixel value of the target pixelis properly corrected also in the imaging blur compensating unit 23.Then, to avoid confusion, during explanation of the imaging blurcompensating unit 23, each pixel value at the stage when it is input toeach functional block will be called an input pixel value, and a pixelvalue at the stage when it is output from each functional block will becalled an output pixel value. Further, there is a case that, withrespect to the same pixel, a plurality of different pixel values areinput from a plurality of preceding functional blocks to a certainfunction block. In such a case, the pixel value closer to an original(mainly, a pixel value before correction) will be called an input pixelvalue, and the other pixel values will be called output pixel values ofa subsequent functional block. For example, although the details will bedescribed later, different values are supplied as pixel values of thetarget pixel from the ALTI unit 81 and the external filter unit 22 tothe subtractor 82. Therefore, the pixel value supplied from the externalfilter unit 22 will be called an input pixel value, and the pixel valuesupplied from the ALTI unit 81 will be called an output pixel value ofthe ALTI unit 81.

The ALTI unit 81 determines a correction amount according to the travelspeed in the target pixel supplied from the imaging blur characteristicdetector 12, adds the correction amount to the input pixel value of thetarget pixel, and supplies the added value as an output pixel value ofthe target pixel to the subtractor 82. The more details of the ALTI unit81 will be described later with reference to FIG. 24.

The subtractor 82 calculates the difference between the output pixelvalue of the target pixel of the ALTI unit 81 and the input pixel valueof the target pixel, and supplies the differential value (hereinbelow,called a target pixel differential value) to the gain controller 83.

The gain controller 83 adjusts the target pixel differential valuesupplied from the subtractor 82 in accordance with the travel speed inthe target pixel supplied from the imaging blur characteristic detector12, and supplies the adjusted target pixel differential value as a finalcorrection amount for the target pixel to the adder 84.

The adder 84 adds the final correction amount from the gain controller83 to the input pixel value of the target pixel, and outputs the addedvalue as an output pixel value of the target pixel to the outside. Thatis, the output pixel value of the target pixel of the adder 84 is outputas the pixel value of the target pixel finally corrected by the imagingblur suppression compensating unit 23 to the outside.

The details of each of the ALTI unit 81 and the gain controller 83 inthe imaging blur compensating unit 23 will be described in that orderbelow.

First, referring to FIGS. 24 to 29, the details of the ALTI unit 81 willbe described.

FIG. 24 shows an example of a detailed functional configuration of theALTI unit 81.

In an example of FIG. 24, the ALTI unit 81 is configured to have delayunits 91-1 to 91-n (hereinbelow, called DL units 91-1 to 91-ncorresponding to FIG. 24), average value calculators 92 to 94, acorrection amount determining unit 95, and an adder 96.

The detailed functional configuration (connection mode of eachfunctional block) of the ALTI unit 81 and its operation will bedescribed below.

The DL units 91-1 to 91-n are connected in that order. Each of the DLunits 91-1 to 91-n delays each of the pixel values output from apreceding DL unit only by predetermined delay time, and outputs theresultant signal to a subsequent DL unit. The pixel values output fromeach of the DL units 91-1 to 91-(n/2−1) is supplied to an average valuecalculator 93. The pixel values output from the DL units 91-(n/2−1),91-(n/2), and 91-(n/2+1) are supplied to the average value calculator92. The pixel values output from the DL units 91-(n/2+1) to 91-n aresupplied to the average value calculator 94. The pixel value output fromthe DL unit 91-(n/2) is also supplied to the correction amountdetermining unit 95 and the adder 96.

Each pixel value of n pixels arranged successively in the traveldirection (in this case, the lateral direction) using the target pixelas a center are sequentially input from the filter unit 22 to the DLunit 91-1 in the arrangement order of the pixels from right to left.Therefore, after time n times as long as delay time nearly elapses sincethen, the pixel value of each of the n pixels arranged successively inthe lateral direction using the target pixel as a center is output oneby one from each of the DL units 91-1 to 91-n.

In addition, description will be given on assumption that each of pixelvalues at the stage when they are output from each of DL units 91-1 to91-n is input pixel values to the ALTI unit 81.

Concretely, an input pixel value N of the target pixel is output fromthe DL unit 91-(n/2) one by one. The input pixel value of each of then/2−1 pixels arranged successively on the left side of the target pixelis output from each of the DL units 91-1 to 91-(n/2−1). On the otherhand, the input pixel value of each of the n/2−1 pixels arrangedsuccessively on the right side of the target pixel is output from eachof the DL units 91-(n/2+1) to 91-n one by one.

In addition, the number n of DL units 91-1 to 91-n is, though notparticularly limited, the highest value [pixels/frame] of the travelspeed in this case. It is also assumed that the travel speed in thetarget pixel supplied from the imaging blur characteristic detector 12is v [pixels/frame] in a manner similar to the above example.

Therefore, to the average value calculator 92, the input pixel value Nof the target pixel, the input pixel value of the pixel on the left sideof the target pixel, and the input pixel value of the pixel on the rightside of the target pixel are input. Then, the average value calculator92 calculates an average value Na of the input pixel value N of thetarget pixel, the input pixel value of the pixel on the left side of thetarget pixel, and the input pixel value of the pixel on the right sideof the target pixel (hereinbelow, called an average pixel value Na ofthe target pixel), and supplies the average value Na to the correctionamount determining unit 95.

As the details will be described later, a correction amount ADD of thetarget pixel determined by the correction amount determining unit 95 isadjusted by a predetermined adjustment amount c. The adjustment value cis not a fixed value but a variable value determined by a predeterminedprocess (hereinbelow, called an adjustment amount determining process).In the embodiment, in the adjustment amount determining process, theaverage pixel value Na of the target pixel is used for the followingreason. Although the input pixel value N of the target pixel can be usedas it is in the adjustment amount determining process, in this case, ifnoise is included in the target pixel, an image to be processed may bebroken. That is, the reason is to prevent breakage of an image to beprocessed.

To the average value calculator 93, the input pixel values of n/2−1pixels successively arranged on the left side of the target pixel aresupplied. Then, the average value calculator 93 selects k pixels (wherek is about v/2) which is about the half of the travel speed in order inthe left direction of the pixel on the left side of the target pixel,and determines a range including the input pixel values of the selectedk pixels as a calculation range. Then, the average value calculator 93calculates an average value La of the k input pixel values included inthe calculation range (hereinbelow, called the average pixel value La ofthe left pixels) in the supplied n/2−1 input pixel values, and suppliesit to the correction amount determining unit 95.

On the other hand, to the average value calculator 94, the input pixelvalues of the n/2−1 pixels arranged successively on the right side ofthe target pixel are supplied. Then, the average value calculator 94selects k pixels in order in the right direction of the pixel on theright side of the target pixel, and determines a range including theinput pixel values of the selected k pixels as a calculation range.Then, the average value calculator 94 calculates an average value Ra ofthe k input pixel values included in the calculation range (hereinbelow,called the average pixel value Ra of the right pixels) in the suppliedn/2−1 input pixel values, and supplies it to the correction amountdetermining unit 95.

As the details will be described later, the average pixel value La ofthe left pixels and the average pixel value Ra of the right pixels areused for the adjustment amount determining process and a process fordetermining a candidate of the correction amount (hereinbelow, calledcandidate determining process).

That is, in the LTI of the related art disclosed in the above-mentionedJapanese Unexamined Patent Application Publication No. 2000-324364, thedifferential value between the input pixel value of one pixel(hereinbelow, called left pixel) apart from the target pixel only by apredetermined distance in the left direction and the input pixel valueof the target pixel is determined as a first candidate of the correctionamount. Further, the differential value between the input pixel value ofone pixel (hereinbelow, called right pixel) apart from the target pixelonly by a predetermined distance in the right direction and the inputpixel value of the target pixel is determined as a second candidate ofthe correction amount. Then, one of the first and second candidates isdetermined as a correction amount as it is without being adjusted.Consequently, the LTI of the related art has an issue such that if noiseis included in the input pixel value of the left pixel or the rightpixel, the correction amount (two candidates) may not be properlydetermined.

Therefore, to solve the issue, that is, to properly determine candidatesof the correction amount, in the candidate determining process of theembodiment, the input pixel value of one pixel such as a left pixel orright pixel is not simply used, but the average pixel value La of theleft pixels and the average pixel value Ra of the right pixels are used.

However, there is the case that the change direction of each input pixelvalue included in the calculation range is not constant, that is,increases and then decreases, or decreases and then increases,conversely. In other words, there is the case that the polarity of thegradient of a line connecting points indicative of each input pixelvalue included in the calculation range (points 131 to 134 and the likeof FIG. 25 which will be described later) is inverted on a plane usingthe pixel positions in the horizontal direction as the horizontal axisand using the pixel values as the vertical axis (for example, the planeof FIG. 25 which will be described later). In such a case, a new issueoccurs such that even if a simple average value of input pixel valuesincluded in the calculation range is employed as the average pixel valueLa of the left pixels or the average pixel value Ra of the right pixels,a correction amount (candidate) may not be properly determined.

Therefore, to solve the new issue, in the embodiment, each of theaverage value calculators 93 and 94 updates an input pixel value βindicated by a first point after polarity inversion in the input pixelvalues included in the calculation range to a pixel value γ by computingthe right side of the following equation (5) using an input pixel valueα indicated by a second point before polarity inversion. Each of theaverage value calculators 93 and 94 regards the input pixel value of thepixel indicated by the first point as the updated pixel value γ, andcalculates the average pixel value La of left pixels or the averagepixel value Ra of right pixels.

γ=α−H×f(H)  (5)

In the equation (5), as shown in FIG. 25, H denotes the differentialvalue (=α−β) between the pixel value α of the second point (point 133 inthe diagram) before polarity inversion and the pixel value β of thefirst point (point 134 in the diagram) after polarity inversion.

That is, FIG. 25 shows an example of pixel values of 12 pixels arrangedsuccessively in the horizontal direction including the target pixel 131.In FIG. 25, the horizontal axis indicates “pixel position in thehorizontal direction”, and the vertical axis indicates “pixel values”.In the example of FIG. 25, the calculation range of the average valuecalculator 94, that is, the calculation range of the average pixel valueRa of right pixels is a range D including the pixel values α, α, and βindicated by the three points 132 to 133 on the right side of point 131indicative of the target pixel.

From the example of FIG. 25, it is understood that the polarity of thegradient from point 133 to point 134 is determined. To be specific, thepoint 134 is the first point after the polarity inversion, and the point133 is the second point before the polarity determination. Therefore, inthe example of FIG. 25, the average value calculator 94 varies the inputpixel value indicated by the point 134 from the pixel value β to thepixel value γ by assigning and calculating the input pixel value αindicated by the point 133 and the difference value H(=α−β) between theinput pixel value α and the input pixel value β indicated by the point134 to the right side of the equation (5). Then, the average valuecalculator 94 calculates the average pixel value Ra of right pixelsusing the updated pixel value γ as the input pixel value of the pixelindicated by the point 134 in the calculation range D and using theoriginal pixel value α as it is as each of input pixel values of theother points 132 and 133. That is, Ra=(α+α+γ)/3 is computed.

In the embodiment, in computation of the right side of the equation (5),a function having the characteristic like a line 141 of FIG. 26 is usedas the function f(H).

As shown in FIG. 26, in the case where the differential value H betweenthe pixel value α before polarity inversion and the pixel value β afterpolarity inversion is equal to a value H2 or larger, an output of thefunction f(H) is zero. In addition, when the differential value H islarge, it means that the gradient after polarity inversion is sharp.Therefore, in the case where the gradient after polarity inversion issharp to a certain extent or more, that is, in the case where thedifferential value H is the value H2 or larger, the pixel value γupdated by the equation (5) becomes the pixel value α. That is, as shownin FIG. 25, in the case where the gradient after polarity inversion issharp to a certain extent or more, the average pixel value Ra of rightpixels in the calculation range D is calculated using the pixel value αin place of the pixel value β as the input pixel value of the pixelindicated by the point 134 after polarity inversion. That is,Ra=(α+α+α)/3=α is computed and the average pixel value Ra of rightpixels is determined as the pixel value α.

On the other hand, as shown in FIG. 26, in the case where thedifferential value H between the pixel value α before polarity inversionand the pixel value β after polarity inversion is equal to the value H1or less, an output of the function f(H) becomes 1. In addition, when thedifferential value H is small, it means that the gradient after polarityinversion is gentle. Therefore, in the case where the gradient afterpolarity inversion is gentle to a certain extent or more, that is, inthe case where the differential value H is the value H1 or less, thepixel value γ updated by the equation (5) remains the pixel value β.That is, in the case where the gradient after polarity inversion isgentle to a certain extent or more, although not shown, the averagepixel value Ra of right pixels in the calculation range D is calculatedusing the pixel value β as it is as the input pixel value indicated bythe point 134 after polarity inversion. That is, Ra=(α+α+β)/3 iscomputed and the average pixel value Ra of right pixels is determined asthe pixel value {(α+α+β)/3}.

In addition, when the gradient after polarity inversion is gentle to acertain extent or more, the original pixel value β is used as it iswithout updating the pixel value indicated by the point 134 afterpolarity inversion for the following reason. That is, in the case wherethe gradient after polarity inversion is gentle to a certain extent ormore, the possibility that polarity inversion occurs due to noise ishigh. In this case, by obtaining an average without updating the inputpixel values, the appropriate average pixel value Ra of right pixelswithout noise can be obtained.

The case of calculating the average pixel value Ra of right pixels hasbeen described above using the concrete example of FIG. 25. Also in theother cases, for example, in the case of calculating the average pixelvalue La of left pixels, the input pixel value of the pixel indicated bythe point after polarity inversion is updated from the pixel value β tothe pixel value γ similarly by the equation (5).

Referring again to FIG. 24, the number of taps (the number of pixelvalues) used in the case of calculating the average value in each of theabove-described average value calculators 92 to 94 is fixed in the aboveexample. However, it may be varied, for example, according to the travelspeed in the target pixel in the output signal of the imaging blurcharacteristic detector 12. Concretely, for example, it may be varied soas to increase the number of taps (that is, to increase the width ofaverage) as the travel speed increases. The results of the average valuecalculators 92 to 94 using taps of the number according to the travelspeed as described above are used by the correction amount determiningunit 95 which will be described later, thereby enabling a correctionamount for performing higher-precision correction, that is, correctioncapable of further suppressing the imaging blur to be determined.

The correction amount determining unit 95 determines the correctionamount ADD by using the input pixel value N of the target pixel from theDL unit 91-(n/2), the average pixel value Na of the target pixel fromthe average value calculator 92, the average pixel value La of leftpixels from the average value calculator 93, and the average pixel valueRa of right pixels from the average value calculator 94, and supplies itto the adder 96.

Here, the adder 96 adds the correction amount ADD from the correctionamount determining unit 95 to the input pixel value N of the targetpixel from the DL unit 91-(n/2), and supplies the addition result as anoutput pixel value of the target pixel, that is, a corrected pixel valueof the target pixel to the adder 82 on the outside of the ALTI unit 82.

Before explaining an example of a detailed functional configuration ofthe correction amount determining unit 95, the process of the ALTI unit81 will be described with reference to the flowchart of FIG. 27.

In step S21, the ALTI unit 81 sets a target pixel.

In step S22, the DL units 91-1 to 91-n of the ALTI unit 81 obtain npieces of neighboring input pixel values around the input pixel value Nof the target pixel as a center.

In step S23, the average value calculator 92 of the ALTI unit 81calculates the average pixel value Na of the target pixel and suppliesit to the correction amount determining unit 95 as described above.

In step S24, the average value calculator 93 in the ALTI unit 82calculates the average pixel value La of left pixels and supplies it tothe correction amount determining unit 95 as described above.

In step S25, the average value calculator 94 in the ALTI unit 82calculates the average pixel value Ra of right pixels and supplies it tothe correction amount determining unit 95 as described above.

In addition, as obvious from FIG. 24, each of the average valuecalculators 92 to 94 executes the process independently of others.Therefore, the order of processes in the steps S23 to S25 is not limitedto the example of FIG. 27 but may be an arbitrary order. That is, inreality, the processes in the steps S23 to S25 are executed in paralleland independently of others.

In step S26, the correction amount determining unit 95 in the ALTI unit82 determines two candidates ADDL and ADDR of the correction amount byusing the input pixel value N of the target pixel from the DL unit91-(n/2), the average pixel value La of left pixels from the averagevalue calculator 93, and the average pixel value Ra of right pixels fromthe average value calculator 94. That is, the process of step S26 is theabove-described candidate determining process. The candidates ADDL andADDR of the correction amount are respective output signals fromsubtractors 101 and 102 which will be described later. In addition, thedetails of the candidate determining process in step S26 and thecandidates ADDL and ADDR of the correction amount will be describedlater.

In step S27, the correction amount determining unit 95 determines theadjustment amount c by using the average pixel value Na of the targetpixel from the average value calculator 92, the average pixel value Laof the left pixels from the average value calculator 93, and the averagepixel value Ra of the right pixels from the average value calculator 94.That is, the process in step S27 is the above-described adjustmentamount determining process. The adjustment amount c denotes an outputsignal of the adjustment amount value calculator 109 which will bedescribed later. The details of the adjustment amount determiningprocess in step S27 and the adjustment amount c will be described later.

In addition, as the details will be described later, in reality, theprocesses in the steps S26 and S27 are executed in parallel andindependently of each other. That is, the order of the processes in thesteps S26 and S27 is not limited to the example of FIG. 27 but may be anarbitrary order.

In step S28, the correction amount determining unit 95 adjusts each ofthe values of the candidates ADDL and ADDR by using the adjustmentamount c. In the following, the process in the step S28 will be calledthe adjusting process. The details of the adjusting process will bedescribed later.

In step S29, the correction amount determining unit 95 determines(selects), as the correction amount ADD, predetermined one of thecandidates ADDL and ADDR whose values are adjusted by the adjustmentamount c and 0 in accordance with a predetermined discriminationcondition, and supplies it to the adder 96. In the following, theprocess in the step S29 will be called a correction amount selectingprocess. The details (including the discrimination condition) of thecorrection amount selecting process will be described later.

In step S30, the adder 96 in the ALTI unit 81 adds the correction amountADD to the input pixel value N of the target pixel and outputs theresultant addition value as the output pixel value of the target pixelto the adder 82 on the outside.

In step S31, the ALTI unit 81 determines whether the process has beenfinished on all of the pixels or not.

In the case where it is determined in the step S31 that the process hasnot been finished yet on all of the pixels, the process is returns tothe step S21, and the following processes are repeated. Specifically,another pixel is set as the target pixel, the correction amount ADD isadded to the input pixel value N of the target pixel, and the resultantaddition value is output as the output pixel value of the target pixelto the adder 82 on the outside. Naturally, each of the pixel value N andthe correction amount ADD often varies among the pixels.

After all of the pixels are set as the target pixels and theabove-mentioned loop process in the steps S21 to S31 is repeatedlyexecuted for each of the set target pixels, it is determined in step S31that the process on all of the pixels is finished and the process of theALTI unit 81 is finished.

In addition, since the ALTI unit 81 is a component of the imaging blursuppression processor 13 in FIG. 13, the above-mentioned process of theALTI unit 81 in FIG. 27 is executed as a part of the above-mentionedprocess of the step S4 in FIG. 15.

As described above, the correction amount determining unit 95 executesthe processes in the steps S26 to S29. In the following, referring againto FIG. 24, while describing an example of the detailed functionalconfiguration of the correction amount determining unit 95, the detailsof the processes in the steps S26 to S29 will be also described.

As shown in FIG. 24, the correction amount determining unit 95 isprovided with the adders 101 and 102 in order to execute theabove-described candidate determining process in the step S26 in FIG.27. In other words, a candidate determining unit 121 constructed by thesubtractors 101 and 102 executes the candidate determining process inthe step S26.

The subtractor 101 calculates the differential value (=La−N) between theaverage pixel value La of left pixels from the average value calculator93 and the input pixel value N of the target pixel from the DL unit91-(n/2) and supplies the differential value as the candidate ADDL ofthe correction amount to a multiplier 110.

In addition, as will be described later, in the case where the candidateADDL of the correction amount is determined as the correction amount ADDwithout being adjusted (multiplied with the adjustment amount c=1), theadder 96 adds the correction amount ADD (=La−N) to the input pixel valueN of the target pixel and the resultant addition value (=La) is outputto the outside. That is, in the case where the candidate ADDL (=La−N) ofthe correction amount is used as it is as the correction amount ADD, thepixel value of the target pixel is corrected (replaced) from theoriginal pixel value N to the average pixel value La of left pixels.

The subtractor 102 calculates the differential value (=Ra−N) between theaverage pixel value Ra of right pixels from the average value calculator94 and the input pixel value N of the target pixel from the DL unit91-(n/2) and supplies the differential value as the candidate ADDR ofthe correction amount to a multiplier 111.

In addition, as will be described later, in the case where the candidateADDR of the correction amount is determined as the correction amount ADDwithout being adjusted (multiplied with the adjustment amount c=1), theadder 96 adds the correction amount ADD (=Ra−N) to the input pixel valueN of the target pixel and the resultant addition value (=Ra) is outputto the outside. That is, in the case where the candidate ADDR (=Ra−N) ofthe correction amount is used as it is as the correction amount ADD, thepixel value of the target pixel is corrected (replaced) from theoriginal pixel value N to the average pixel value Ra of right pixels.

In addition, as shown in FIG. 24, the correction amount determining unit95 is provided with components from a subtractor 103 to an adjustmentamount value calculator 109 in order to execute the above-mentionedadjustment amount determining process in the step S27 in FIG. 27. Inother words, an adjustment amount determining unit 122 constructed bythe subtractor 103 to the adjustment amount determining unit 109executes the adjustment amount determining process in the step S27.

The subtractor 103 calculates the differential value (=Na−La) betweenthe average pixel value Na of the target pixel from the average valuecalculator 92 and the average pixel value La of left pixels from theaverage value calculator 93 and supplies the differential value to anadder 105.

A subtractor 104 calculates the differential value (=Na−Ra) between theaverage pixel value Na of the target pixel from the average valuecalculator 92 and the average pixel value Ra of right pixels from theaverage value calculator 94 and supplies the differential value to theadder 105.

The adder 105 calculates the sum of output signals of the subtractors103 and 104 and outputs the calculation result to an ABS unit 106.

The ABS unit 106 calculates an absolute value b of the output signal ofthe adder 105 and supplies the absolute value b to a divider 108.

In other words, in a plane using the pixel values as the vertical axisand using pixel positions in the horizontal direction as the horizontalaxis, a quadratic differential value at a second point on a linesequentially connecting a first point indicative of the average pixelvalue La of left pixels, the second point indicative of the averagepixel value Na of the target pixel, and a third point indicative of theaverage pixel value Ra of right pixels is computed by the subtractors103 and 104 and the adder 105. The absolute value b of the quadraticdifferential value is computed by the ABS unit 106, and supplied to thedivider 108. Therefore, the absolute value b output from the ABS unit106 will be called the quadratic differential absolute value b below.

In the foregoing plane, in the case where a straight line connecting thefirst point indicative of the average pixel value La of left pixels andthe third point indicative of the average pixel value Ra of right pixelsis used as a boundary line, the quadratic differential absolute value bis a value indicative of the distance of the second point indicative ofthe average pixel value Na of the target pixel from the boundary line inthe vertical axis direction.

Consequently, the correction amount determining unit 95 adjusts each ofthe values of the candidates ADDL and ADDR of the correction amount inaccordance with the magnitude of the quadratic difference absolute valueb, and determines one of the adjusted candidates ADDL and ADDR as thecorrection amount ADD. That is, the adder 96 outputs, as the outputpixel value of the target pixel, the addition value between the inputpixel value N of the target pixel and the correction amount ADD adjustedaccording to the magnitude of the quadratic differential absolute valueb. As a result, an edge portion in the output signal (the frame to beprocessed) of the adder 96 can be made gentle.

However, even if the quadratic differential absolute values b are thesame, when the absolute value h of the difference between the averagepixel value La of left pixels and the average pixel value Ra of rightpixels, that is, the distance h between the first and third points inthe vertical axis direction in the above-described plane (hereinbelow,called height h) varies, the meaning of the magnitude of the quadraticdifferential absolute value b varies. Specifically, even if thequadratic differential absolute value b is the same, in the case wherethe magnitude is much smaller than the height h, in other words, in thecase where a division value (=b/h) obtained by dividing the quadraticdifferential value b by the height h is small, it can be determined thatthe possibility of occurrence of noise around the target pixel is high.On the other hand, even if the quadratic differential absolute values bare the same, in the case where the magnitude is not so small ascompared with the height h, in other words, in the case where theabove-mentioned division value (=b/h) has a certain magnitude or more,it can be determined that the possibility of occurrence of noise aroundthe target pixel is low.

Therefore, if the values of the candidates ADDL and ADDR are adjustedsimply according to the magnitude of the quadratic differential absolutevalue b, the correction amount ADD of the input pixel value N of thetarget pixel becomes the same value irrespective of whether noise occursor not. A new issue occurs such that the input pixel value N of thetarget pixel may not be properly corrected.

Then, to solve the new issue, the adjustment amount determining unit 122of the correction amount determining unit 95 of the embodiment isprovided with the components from the above-mentioned subtractor 103 tothe ABS unit 106 and, in addition, a difference absolute valuecalculator 107, the divider (b/h calculator) 108, and the adjustmentamount value calculator 109.

The differential absolute value calculator 107 calculates the differencevalue between the average pixel value La of left pixels from the averagevalue calculator 93 and the average pixel value Ra of right pixels fromthe average value calculator 94, further calculates the absolute value h(h=|La−Na|) of the difference value, that is, the above-mentioned heighth, and supplies the height h to the divider 108.

The divider 108 divides the quadratic differential absolute value b fromthe ABS unit 106 by the height h from the difference absolute valuecalculator 107 and provides the division value (=b/h) to the adjustmentamount calculator 109. That is, the division value (=b/h) can be said asa value obtained by normalizing the quadratic differential absolutevalue b by the height h. Therefore, the division value (=b/h) will becalled a normalized quadratic differential value (=b/h).

The adjustment amount calculator 109 calculates the adjustment amount cfor the candidates ADDL and ADDR on the basis of the normalizedquadratic differential value (=b/h) from the divider 108, and suppliesit to the multipliers 110 and 111.

Specifically, for example, the adjustment amount calculator 109 holds afunction of the characteristic expressed by a curve 151 in FIG. 28,assigns the normalized quadratic differential value (=b/h) from thedivider 108 as an input parameter to the function, and supplies anoutput of the function (output of FIG. 28) as the adjustment amount c tothe multipliers 110 and 111.

That is, as easily understood from the shape of the curve 151 of FIG.28, when the normalized quadratic differential value (=b/h) is smallerthan a predetermined value b1, the possibility of noise is high, and theadjustment amount c (output) becomes zero. In this case, as will bedescribed later, the candidates ADDL and ADDR are adjusted by beingmultiplied with zero as the adjustment amount c, so that each of theadjusted candidates ADDL and ADDR becomes zero. Therefore, thecorrection amount ADD also becomes zero, and the input pixel value N ofthe target pixel is not corrected.

Further, when the normalized quadratic differential value (=b/h) exceedsthe predetermined value b1 and increases, the adjustment amount c(output) also increases gradually. In this case, as will be describedlater, each of the candidates ADDL and ADDR is adjusted by beingmultiplied with the adjustment amount c which is less than 1, so thateach of the adjusted candidates ADDL and ADDR becomes smaller than theoriginal value. Therefore, the correction amount ADD becomes one of thecandidates ADDL and ADDR which became smaller than the original values.The corrected pixel value of the target pixel becomes larger than theaverage pixel value La of left pixels or smaller than the average pixelvalue Ra of right pixels.

Further, when the normalized quadratic differential value (=b/h) becomesa predetermined value b2 or larger, after that, the adjustment amount c(output) becomes 1. In this case, as will be described later, each ofthe candidates ADDL and ADDR is adjusted by being multiplied with 1 asthe adjustment amount c, so that each of the adjusted candidates ADDLand ADDR remains the original value (that is, not adjusted). Therefore,the correction amount ADD becomes one of the candidates ADDL and ADDRremaining as the original values. As described above, the correctedpixel value of the target pixel becomes the average pixel value La ofleft pixels or the average pixel value Ra of right pixels.

As described above, in the present embodiment, the adjustment amount cis determined using the function of the characteristic expressed by theline 151 of FIG. 28, to which the normalized quadratic differentialvalue (=b/h) is input as a parameter. Consequently, by adjusting thecorrection amount ADD with the adjustment amount c (to be accurate, byadjusting the candidates ADDL and ADDR of the correction amount), theedge portion in the output signal (the frame to be processed) of theadder 96 can be made gentle. Specifically, in the LTI of the relatedart, the pixel value of the target pixel is corrected by switching(simple replacement of the pixel value) of a hard switch. There isconsequently an issue that the edge portion in the output signal may notbe made gentle. However, by employing the ALTI unit 81 of theembodiment, the issue can be solved.

Referring again to FIG. 24, the detailed description of the correctionamount determining unit 95 will be continued. Specifically, thecorrection amount determining unit 95 is provided with the multipliers110 and 111 in order to execute the adjusting process in theabove-mentioned step S28 in FIG. 27. In other words, an adjusting unit123 constructed by the multipliers 101 and 111 executes the adjustingprocess in the step S28.

The multiplier 110 multiplies the candidate ADDL from the subtractor 101with the correction amount c from the adjustment amount calculator 109,and supplies the resultant multiplied value as the adjusted candidateADDL to a discriminator 113.

The multiplier 111 multiplies the candidate ADDR from the subtractor 102with the correction amount c from the adjustment amount calculator 109,and supplies the resultant multiplied value as the adjusted candidateADDR to the discriminator 113.

In addition, the correction amount determining unit 95 is also providedwith a fixed value generator 112 and the discriminator 113 in order toexecute the above-mentioned correction amount selecting process in thestep S29 in FIG. 27. In other words, a correction amount selecting unit124 constructed by the fixed value generator 112 and the discriminator113 executes the correction amount selecting process in the step S29.

In the embodiment, the fixed value generator 112 always generates “0” asshown in FIG. 24 and supplies it to the discriminator 113.

To the discriminator 113, output signals of the subtractors 103 and 104,the adder 105, the multipliers 110 and 111, and the fixed valuegenerator 112 are supplied. The discriminator 113 selects (determines),as the correction amount ADD, predetermined one of “0” from the fixedvalue generator 112, the corrected candidate ADDL from the multiplier110, and the corrected candidate ADDR from the multiplier 111 on thebasis of a predetermined selecting condition using output signals of thesubtractors 103 and 104 and the adder 105, and supplies it to the adder96.

Concretely, for example, in the above-mentioned plane using the pixelvalues as the vertical axis and using pixel positions in the horizontaldirection as the horizontal axis, a straight line connecting a firstpoint indicative of the average pixel value La of left pixels and athird point indicative of the average pixel value Ra of right pixels isset as a boundary line. The selecting condition of the embodiment isassumed to be specified that the corrected candidate ADDR is selected asthe correction amount ADD in the case where the change direction of theboundary line is an upward direction and a second point indicative ofthe average pixel value Na of the target pixel is disposed on the upperside of the boundary line. On the contrary, the selecting condition ofthe embodiment is assumed to be specified that the corrected candidateADDL is selected as the correction amount ADD in the case where thechange direction of the boundary line is an upward direction and thesecond point is disposed on the lower side of the boundary line.

In this case, the discriminator 113 can recognize the change directionof the boundary line and the positional relation between the boundaryline and the second point on the basis of the output signals of thesubtractors 103 and 104 and the adder 105.

Then, for example, in the case where the discriminator 113 recognizesthat the change direction of the boundary line is an upward directionand the second point is disposed on the upper side of the boundary lineon the basis of the output signals of the subtractors 103 and 104 andthe adder 105, the discriminator 113 selects (determines) the correctedcandidate ADDR from the multiplier 111 as the correction amount ADD andsupplies it to the adder 96.

On the other hand, for example, in the case where the discriminator 113recognizes that the change direction of the boundary line is an upwarddirection and the second point is disposed on the lower side of theboundary line on the basis of the output signals of the subtractors 103and 104 and the adder 105, the discriminator 113 selects (determines)the corrected candidate ADDL from the multiplier 110 as the correctionamount ADD and supplies it to the adder 96.

It is also assumed that, in the case where the target pixel ispositioned in a location other than an edge portion for example,selection of 0 as the correction amount ADD is specified as a selectingcondition of the embodiment. In this case, for example, when thediscriminator 113 recognizes that all of the output signals of thesubtractors 103 and 104 and the adder 105 are almost zero, that is, whenthe average pixel value La of left pixels, the input pixel value N ofthe target pixel, and the average pixel value Rc of right pixels arealmost the same or the like, the discriminator 113 recognizes that thetarget pixel is positioned in a location other than the edge portion,selects (determines) “0” from the fixed value generator 112 as thecorrection amount ADD, and supplies it to the adder 96.

As an embodiment of the ALTI unit 81, the ALTI unit 81 having thefunctional configuration of FIG. 24 has been described above. As long asprocesses equivalent to the above-described series of processes can beexecuted, any functional configuration may be used as the functionalconfiguration of the ALTI unit 81. Concretely, for example, the ALTIunit 81 may have a functional configuration shown in FIG. 29. That is,FIG. 29 shows an example of a detailed functional configurationdifferent from FIG. 24 of the ALTI unit 81.

In the example of FIG. 29, the ALTI unit 81 is configured to have amasking signal generator 161, an LTI processing unit 162, and anaveraging unit 163.

The masking signal generator 161 receives the output signal of thefilter unit 22 as own input signal and sequentially sets, as the targetpixel, each of the pixels constructing the frame to be processed in theinput signal. The masking signal generator 161 searches pixels on theleft and right sides of the target pixel by the number of pixelscorresponding to the half of the travel speed from the target pixel, andperforms masking process on each signal indicative of the pixel valuesof the number of pixels corresponding to the travel speed. The travelspeed of the target pixel is supplied from the imaging blurcharacteristic detector 12 as described above. The masked signals aresupplied from the masking signal generator 161 to the LTI processingunit 162.

The LTI processing unit 162 performs the LTI process on each of themasked signals and supplies, as an output signal, the resultant signalto the averaging unit 163.

The averaging unit 163 averages signals of the number corresponding tothe number of search times in the masking signal generator 161, in theoutput signals of the LTI processing unit 162 and supplies the resultantsignal as an output signal of the ALTI unit 81 to the adder 82 on theoutside.

Referring to FIGS. 24 to 29, the details of the ALTI unit 81 in theimaging blur compensating unit 23 in FIG. 23 have been described above.

Next, referring to FIGS. 30 and 31, the details of the gain controller83 in the imaging blur compensating unit 23 in FIG. 23 will bedescribed.

FIG. 30 shows an example of the detailed functional configuration of thegain controller 83. FIG. 31 shows the characteristic of an adjustmentamount determining unit 171 which will be described later in the gaincontroller 83 in FIG. 30.

In the example of FIG. 30, the gain controller 83 is configured to havethe adjustment amount determining unit 171 and a multiplier 172.

The adjustment amount determining unit 171 holds a function expressed bya curve 181 in FIG. 31, assigns travel speed in the target pixelsupplied from the imaging blur characteristic detector 12 as an inputparameter to the function, and supplies an output of the function(output of FIG. 31) as an adjustment amount to the multiplier 172. Inother words, the adjustment amount determining unit 171 has thecharacteristic expressed by the curve 181 of FIG. 31.

To the multiplier 172, an adjustment amount from the adjustment amountdetermining unit 171 and, in addition, an output signal of the adder 82are also supplied. As obvious from the above-mentioned functionalconfiguration of FIG. 23, an output signal of the adder 82 is acandidate of a final correction amount added to an input pixel value ofa target pixel for the imaging blur compensating unit 23, in the adder84. Specifically, the multiplier 172 multiplies the candidate of thefinal correction amount with the adjustment amount from the adjustmentamount determining unit 171 and supplies the resultant multiplied value,as the final adjustment amount, to the adder 84.

That is, as easily understood from the shape of the line 181 in FIG. 31and the functional configuration of FIG. 23 of the imaging blurcompensating unit 23, the gain controller 83 controls so that theprocess result (hereinafter called as ALTI) of the ALTI unit 81 does notexert much influence on the final correction amount of the pixel valueof the target pixel when the travel speed is low. When the travel speedis low, deterioration in the gain due to imaging blur is small, and itis sufficient to increase the attenuated gain by the filter unit 22 inFIGS. 17 and 20. That is, it is sufficient to output the output signalof the filter unit 22 as a final output signal of the imaging blurcompensating unit 23 without performing much correction on the outputsignal.

Referring to FIGS. 17 to 31, an example of the imaging blur suppressionprocessor 13 in the video signal processor 4B in FIG. 13 has beendescribed above.

However, the functional configuration of the imaging blur suppressionprocessor 13 is not limited to the above-mentioned example of FIG. 17,but may be variously modified. Concretely, for example, FIGS. 32 and 33show two examples of the functional configuration of the imaging blursuppression processor 13 to which the present invention is applied, andthe two examples are different from the example of FIG. 17.

In the example of FIG. 32, in a manner similar to the example of FIG.17, the imaging blur suppression processor 13 is configured to have thehigh frequency component removing unit 21, the filter unit 22, and theimaging blur compensating unit 23.

Also in the example of FIG. 32, in a manner similar to the example ofFIG. 17, an output signal of the interpolation section 45 is supplied asan input signal to the imaging blur suppression processor 13 to the highfrequency component removing unit 21. An output signal of the imagingblur characteristic detector 12 is supplied to the filter unit 22 andthe imaging blur compensating unit 23.

However, in the example of FIG. 32, an output signal of the highfrequency component removing unit 21 is supplied to the imaging blurcompensating unit 23. An output signal of the imaging blur compensatingunit 23 is supplied to the filter unit 22. An output signal of thefilter unit 22 is output as an output signal indicative of the finalprocess result of the imaging blur suppression processor 13 to theoutside.

In other words, in the example of FIG. 32, the disposing positions ofthe filter unit 22 and the imaging blur compensating unit 23 areopposite to those in the example of FIG. 17. That is, the order ofdisposing positions of the filter unit 22 and the imaging blurcompensating unit 23 (the process order) is not particularly limited.Any of the units may be disposed first.

Further, in the example of FIG. 33, like in the examples of FIGS. 17 and32, the imaging blur suppression processor 13 is provided with the highcomponent removing unit 21, the filter unit 22, and the imaging blurcompensating unit 23 and further, in addition to the functional blocks,is also provided with an adder 24.

Also in the example of FIG. 33, like in the examples of FIGS. 17 and 32,an output signal of the interpolation section 45 is supplied as an inputsignal for the imaging blur suppression processor 13 to the highfrequency component removing unit 21. Further, an output signal of theimaging blur characteristic detector 12 is supplied to each of thefilter unit 22 and the imaging blur compensating unit 23.

However, in the example of FIG. 33, an output signal of the highfrequency component removing unit 21 is supplied to each of the filterunit 22 and the imaging blur compensating unit 23. Output signals of thefilter unit 22 and the imaging blur compensating unit 23 are supplied tothe adder 24. The adder 24 adds the output signal of the filter unit 22and the output signal of the imaging blur compensating unit 23, andoutputs the resultant addition signal as an output signal indicative ofthe final process result of the imaging blur suppression processor 13 tothe outside.

In other words, the filter unit 22 and the imaging blur compensatingunit 23 are arranged in series in the examples of FIGS. 17 and 32 butare arranged in parallel in the example of FIG. 33. That is, the filterunit 22 and the imaging blur compensating unit 23 may be arranged inseries or in parallel. However, if both of the filter unit 22 and theimaging blur compensating unit 23 use a line memory, by arranging thefilter unit 22 and the imaging blur compensating unit 23 in parallel asshown in the example of FIG. 33, the line memory can be shared. As aresult, an effect such that the circuit scale (by the amount of the linememory) can be reduced is produced.

As described above, at the time of reducing a blur of a moving body atthe time of image capturing (imaging blur) by image process, in theconventional technique, the process is performed uniformly irrespectiveof the stationary state and the degree of the blur amount. In contrast,in the present invention, for example, by using the above-mentionedimaging blur suppression processor 13, a travel vector (travel speed) iscalculated, and an enhancement amount is changed according to the stateof a moving image. Thus, without making ringing occur, the blur can bereduced. Further, in the LTI of the related art, the signal was switchedby the hard switch, so that a processed image is often broken. However,the above-mentioned imaging blur suppression processor 13 has the ALTIunit 81 as a component. Consequently, a signal can be switched bysoftware and, as a result, breakage of the processed image can besuppressed.

In addition, in the above-described example, for simplicity ofexplanation, the direction of the travel vector (travel direction) isthe horizontal direction. However, even when the travel direction isanother direction, the imaging blur suppression processor 13 canbasically perform similar processes as the series of processes describedabove. Specifically, regardless of the travel direction, the imagingblur suppression processor 13 can similarly correct the pixel value ofthe target pixel so as to suppress the imaging blur. Concretely, forexample, it is sufficient for the ALTI unit 81 in the functionalconfiguration of FIG. 24 to enter the pixel values of n pixels arrangedsuccessively in the travel direction (for example, the verticaldirection) using the target pixel as a center in the arrangement orderto the DL unit 91-1. In the other functional blocks as well, operationsare similarly performed.

Incidentally, in the above-described example, at the time of correctingthe pixel values, the imaging blur suppression processor 13 uses thetravel speed (the absolute value of the travel vector) as a parameter.However, other than the travel speed, as long as the parameter shows thecharacteristic of an imaging blur, an arbitrary parameter can be used.

Concretely, for example, the imaging blur suppression processor 13 canuse, as a parameter showing the characteristic of an imaging blur,shutter speed of a camera at the time of capturing a moving image to beprocessed. The reason is that, for example, as shown in FIG. 34, whenthe shutter speed varies, the degree of an imaging blur also varies onlyby the amount of time Ts in the diagram.

Specifically, in FIG. 34, the upper diagram shows the case where theshutter speed is 1/30 second which is the same as the frame speed. Thelower diagram shows the case of the shutter speed of ( 1/30−Ts) secondthat is faster than the frame speed. In both of the diagrams of FIG. 34,the horizontal axis expresses the time base, and the vertical axisexpresses the ratio of shutter open time. For example, the ratio of theshutter open time is expressed as (Ts/Vs)×100[%] where the shutter speedis Va [seconds] (Va is an arbitrary value of 0 or larger), the ratio offirst time when the shutter is open is set as 0%, the ratio of secondtime after lapse of V [seconds] from the first time and the shutter isclosed is set as 100%, and time from the first time to present time isexpressed as Ta [seconds] (Ta is an arbitrary positive value from 0 ormore to V or less). In this case, in the vertical axis of the diagramsin FIG. 23, the value which is in contact with the time base is 100[%],and the maximum value (the highest value on each of straight lines) is0[%]. That is, the ratio of the shutter open time increases toward thebottom in the vertical axis of the diagrams in FIG. 34.

It is now assumed that one detecting element in a camera corresponds toa pixel in a frame, for example. In this case, as shown in the upperdiagram of FIG. 34, when the shutter speed is 1/30 second, an integratedvalue of light incident in 1/30 second in which the shutter is open isoutput as a pixel value of the corresponding pixel from one detectingelement in the camera. On the other hand, when the shutter speed is (1/30−Ts) second, an integrated value of light incident in ( 1/30−Ts)second in which the shutter is open is output as a pixel value of thecorresponding pixel from one detecting element in the camera.

That is, the shutter speed corresponds to light accumulation time(exposure time) in a detecting element. Therefore, for example, when amoving object crossing in front of a predetermined detecting elementexists in a real space, light different from light corresponding to theobject, for example, light of the background incident on the detectingelement at the shutter speed of 1/30 second is larger than that at theshutter speed of ( 1/30−Ts) second only by the amount of time Ts[second]. The ratio that the light accumulation value of the backgroundor the like different from the object mixed in the pixel value outputfrom one detecting element at the shutter speed of 1/30 second is higherthan that at the shutter speed of ( 1/30−Ts) second. As a result, thedegree of an imaging blur increases.

The above is summarized as follows. The lower the shutter speed becomes,the higher the degree of image blur becomes. That is, it can betherefore said that the shutter speed expresses a characteristic of animaging blur. Therefore, the shutter speed can be used as a parameterexpressing a characteristic of an imaging blur as well as the travelspeed.

In addition, in the case where the shutter speed is used as a parametershowing a characteristic of an imaging blur, for example, the imagingblur characteristic detector 12 in FIG. 13 can detect the shutter speedof each frame by analyzing header information added to a moving image(data) supplied from the interpolation section 45 and the like, andsupply the shutter speed as a parameter expressing a characteristic ofthe imaging blur to the imaging blur suppression processor 13. Theimaging blur suppression processor 13 can properly correct each pixelvalue by executing, for example, the above-mentioned series of processesusing the shutter speed in place of the travel speed. The functionalconfiguration of the imaging blur suppression processor 13 in the caseof using the shutter speed can be basically the same as that in the caseof using the travel speed. That is, the imaging blur suppressionprocessor 13 described with reference to FIGS. 17 to 31 can properlycorrect each pixel value by executing the above-mentioned series ofprocesses using the shutter speed as a parameter value.

The video signal processor 4B having the configuration shown in FIG. 13has been described above as an example of the video signal processor ofthe embodiment. The video signal processor of the embodiment is notlimited to the example of FIG. 13 but may have other variousconfigurations.

Specifically, for example, each of FIGS. 35 to 38 is a block diagram ofa part of a video signal processor according to a modified example ofthe embodiment.

For example, a video signal processor of FIG. 35 is configured to have,like the video signal processor 4B of FIG. 13, the interpolation section45, the imaging blur characteristic detector 12, and the imaging blursuppression processor 13.

However, in the video signal processor of FIG. 35, an object of thecorrection process of the imaging blur suppression processor 13 is amoving image which is input to the video signal processor, that is, amoving image before it is subject to the high frame rate convertingprocess of the interpolation section 45. Consequently, the imaging blurcharacteristic detector 12 detects the value of a parameter showing acharacteristic of the imaging blur in the moving image prior to the highframe rate converting process of the interpolation section 45 andsupplies the detection result to the imaging blur suppression processor13.

Therefore, as the image process of the video signal processor of FIG.35, the processes in the steps S1, S3, S4, S2, and S5 in the imageprocesses in FIG. 15 are executed in that order.

In addition, for example, a video signal processor of FIG. 36 isconfigured to have, like the video signal processor 4B of FIG. 13 andthe video signal processor of FIG. 35, the interpolation section 45, theimaging blur characteristic detector 12, and the imaging blursuppression processor 13.

In the video signal processor of FIG. 36, an object of the correctionprocess of the imaging blur suppression processor 13 is a moving imageobtained by performing the high frame rate converting process of theinterpolation section 45 on the input moving image like in the videosignal processor 4B of FIG. 13. Consequently, the imaging blursuppression processor 13 performs a correcting process on the movingimage subjected to the high frame rate converting process.

However, the imaging blur characteristic detector 12 in the video signalprocessor of FIG. 36 detects a parameter showing a characteristic of theimaging blur in the input moving image, that is, in the moving imageprior to the high frame rate converting process of the interpolationsection 45 and supplies the detection result to the imaging blursuppression processor 13. That is, the imaging blur suppressionprocessor 13 of the video signal processor of FIG. 36 corrects eachpixel value using the value of the parameter detected in the movingimage prior to the high frame rate converting process.

Therefore, as the image process of the video signal processor of FIG.36, the processes executed in a flow similar to that of the imagingprocess of FIG. 15, that is, the processes in the steps S1, S2, S3, S4,and S5 are executed in that order. However, the process in the step S3is a process of “detecting the value of a parameter showing acharacteristic of the imaging blur from a moving image prior to the highframe rate converting process, that is, from each of frames constructinga moving image entered by the process in the step S1”.

In contrast to the video signal processors of FIGS. 35 and 36, each ofvideo signal processors of FIGS. 37 and 38 is configured to have theinterpolation section 45 and the imaging blur suppression processor 13and does not include the imaging blur characteristic detector 12 as acomponent.

Specifically, as shown in FIGS. 37 and 38, the imaging blurcharacteristic detector 12 is provided together with a superimposingunit 221 in another video signal processor 211 (hereinbelow, called animage signal generating apparatus 211 as described in the drawing). Amoving image entered to the image signal generating apparatus 211 issupplied to the imaging blur characteristic detector 12 and thesuperimposing unit 221. The imaging blur characteristic detector 12detects the value of a parameter expressing a characteristic of animaging blur from the moving image and supplies it to the superimposingunit 221. The superimposing unit 221 superimposes the value of theparameter indicative of the characteristic of the imaging blur on themoving image and outputs a resultant signal.

Therefore, to the video signal processor of FIG. 37 and the video signalprocessor of FIG. 38, the moving image (signal) on which the value ofthe parameter expressing the characteristic of the imaging blur issuperimposed is supplied from the image signal generating apparatus 211.

Then, for example, in the video signal processor of FIG. 37, the imagingblur suppression processor 13 separates the value of the parameterexpressing the characteristic of the imaging blur and the moving imagefrom each other, and corrects each of the pixel values on the basis ofthe separated value of the parameter expressing the characteristic ofthe imaging blur with respect to each of the frames constructing theseparated moving image.

Next, the interpolation section 45 performs the high frame rateconverting process on the moving image corrected by the imaging blursuppression processor 13 and outputs the resultant moving image, thatis, the moving image converted to the high frame rate and corrected.

Therefore, as the image process of the video signal processor of FIG.37, the processes in the steps S1, S4, S2, and S5 in the image processesin FIG. 15 are executed in that order.

In contrast, for example, in the video signal processor of FIG. 38, theinterpolation section 45 separates the value of the parameter expressingthe characteristic of the imaging blur and the moving image from eachother, performs the high frame rate converting process on the separatedmoving image, and supplies the resultant moving image, that is, themoving image converted to high frame rate to the imaging blursuppression processor 13. At this time, the value of the parametershowing the characteristic of the imaging blur separated by theinterpolation section 45 is also supplied to the imaging blursuppression processor 13.

Next, the imaging blur suppression processor 13 corrects each of thepixel values on the basis of the value of the parameter expressing thecharacteristic of the imaging blur with respect to each of the framesconstructing the moving image converted to high frame rate, and outputsthe resultant moving image, that is, the moving image corrected andconverted to high frame rate.

Incidentally, in the above description on the imaging blur suppressionprocessor 13, for simplicity of explanation, the travel direction (thedirection of the travel vector) is the horizontal direction.Consequently, as a pixel used in the case of performing theabove-described various processes such as the filtering and correctionon the target pixel, pixels neighboring the target pixel in thehorizontal direction are used. In addition, a process using pixelsneighboring the target pixel in a predetermined direction will be calleda process in the predetermined direction. That is, the above-describedexample relates to the process in the horizontal direction.

However, as described above, any direction in a two-dimensional planecan be the travel direction. Naturally, the imaging blur suppressionprocessor 13 can execute the above-mentioned various processes in thesame manner in any direction in a two-dimensional plane such as thevertical direction as the travel direction. However, to perform theprocess in the case where the travel direction is the vertical direction(or the process in the case where the travel direction is an obliquedirection, which is a combination process of the process in the verticaldirection and the process in the horizontal direction), the imaging blursuppression processor 13 has to employ, for example, the configurationof FIG. 39 in place of the above-mentioned configuration of FIG. 17, theconfiguration of FIG. 40 in place of the above-mentioned configurationof FIG. 32, and the configuration of FIG. 41 in place of theabove-mentioned configuration of FIG. 33.

That is, FIGS. 39 to 41 show three examples of the functionalconfiguration of the imaging blur suppression processor 13 to which thepresent invention is applied, which are different from theabove-described examples.

In FIGS. 39, 40, and 41, the same reference numerals as those in FIGS.17, 32, and 33 are designated to corresponding parts (blocks). Theirdescription will be the same so that it will not be repeated.

In the imaging blur suppression processor 13 of the example of FIG. 39,to enable a process in the vertical direction in the configuration ofthe example of FIG. 17, a line memory 261-1 is further provided at theante stage of the filter unit 22, and a line memory 261-2 is provided atthe ante stage of the imaging blur compensating unit 23.

Similarly, in the imaging blur suppression processor 13 of the exampleof FIG. 40, to enable a process in the vertical direction in theconfiguration of the example of FIG. 32, the line memory 261-1 isfurther provided at the ante stage of the imaging blur compensating unit23, and the line memory 261-2 is provided at the ante stage of thefilter unit 22.

On the other hand, in the imaging blur suppression processor 13 in theexample of FIG. 41, to enable a process in the vertical direction in theconfiguration of the example of FIG. 33, only one common line memory 261is further provided at the ante stage of the imaging blur compensatingunit 23 and the filter unit 22.

As described above, by employing the imaging blur suppression processor13 in the example of FIG. 41, as compared with the case employing theconfiguration example of FIG. 39 or 40, the number of line memories canbe reduced without deteriorating the effect of the image blursuppression. That is, by employing the configuration of the example ofFIG. 41 as the configuration of the imaging blur suppression processor13, as compared with the case employing the configuration of the exampleof FIG. 39 or 40, the circuit scale of the imaging blur suppressionprocessor 13 can be reduced and, moreover, the circuit scale of thevideo signal processor 4B of FIG. 13 can be reduced.

In addition, in the embodiment, for example, like in a video signalprocessor 4C shown in FIG. 42, an interpolation position parameterRelpos output from the decoder 47 may be supplied not only to theinterpolator 453 but also to the imaging blur suppression processor 13.In such a configuration, the imaging blur suppression processor 13 canchange the process amount in the imaging blur suppression process inaccordance with the distance of the interpolation position toward avideo image of the closer original frame in each of interpolationframes, which is set by the interpolation section 45. Therefore, thedegree of reducing the imaging blur can be changed according tononuniformity of arrangement of interpolation frames (strength of ajudder). By finely adjusting the degree of suppression of a hold blur ina display image and the degree of suppression of an imaging blur, thepicture quality at the time of watching a movie or the like can beimproved.

In the high frame rate converting process executed in theabove-mentioned various embodiments, the combination of the first framerate (frame frequency) of an input video signal and the second framerate (frame frequency) of an output video signal is not particularlylimited but may be an arbitrary combination. Concretely, for example, 60(or 30) [Hz] is employed as the first frame rate of an input videosignal, and 120[Hz] can be employed as the second frame rate of anoutput video signal. For example, 60 (or 30) [Hz] is employed as thefirst frame rate of an input video signal and 240 [Hz] can be employedas the second frame rate of an output video signal. For example, 50 [Hz]corresponding to the PAL (Phase Alternation by Line) system is employedas the first frame rate of an input video signal, and 100 [Hz] or 200[Hz] can be employed as the second frame rate of an output video signal.For example, 48 [Hz] corresponding to the telecine is employed as thefirst frame rate of an input video signal, and a predetermined frequencyequal to or higher than 48 [Hz] can be employed as the second frame rateof an output video signal.

In addition, by performing the high frame rate converting process in theabove-mentioned various embodiments on the input video signal resultedfrom an existing television system or the like, existing contents can bedisplayed with high grade.

Third Embodiment

A third embodiment of the present invention will now be described.

FIG. 43 shows an example of the configuration of a video signalprocessor (video signal processor 4D) according to the embodiment. Inaddition, the same reference numerals are designated to the samecomponents as those in the foregoing embodiments and their descriptionwill not be repeated.

The video signal processor 4D is obtained by further providing the videosignal processor 4B described in the second embodiment with an overdriveprocessor 10 and performs video signal processes in the interpolationsection 45, the imaging blur suppression processor 13, and the overdriveprocessor 10 in consideration of the reliability in detection of amotion vector mv in the motion vector detector 44. In addition, in thecase of detecting a motion vector also in the imaging blurcharacteristic detector 12, reliability in detection of the motionvector may be considered. In the embodiment, the case where the imagingblur suppression processor 13 and the overdrive processor 10 perform avideo signal process using the motion vector mv detected by the motionvector detector 44 will be described below.

The overdrive processor 10 performs an overdrive process on a videosignal supplied from the imaging blur suppression processor 13 by usingthe motion vector mv detected by the motion vector detector 44.Concretely, the overdrive processor 10 makes the degree of the overdriveprocess rise as the motion vector mv increases, and makes the degree ofthe overdrive process fall off as the motion vector mv decreases. Bysuch an overdrive process, a motion blur and a hold blur in a displayimage can be suppressed.

Here, with reference to FIGS. 45 and 46, the reliability in detection ofthe motion vector mv will be described in detail. FIGS. 44 and 45 showan example of the relation between the presence/absence (MC ON/OFFsignal) in detection of the motion vector mv and the reliability.

In FIG. 44, in the case where the value of MC ON/OFF signal is “0” (ON:the case where the motion vector can be detected) and does not changeand in the case where the value changes from “1” (OFF: the case where nomotion vector can be detected such as the case where the value is out ofthe search range (block matching range) of the motion vector) to “0”,the value of reliability increases to “P (preceding value)+Y (changeamount)”. On the other hand, in the case where the value of the MCON/OFF signal changes from “0” to “1” and in the case where the value is“1” and does not change, the value of reliability decreases to “P-Y”.

With the configuration, for example, as shown in FIG. 45, during aperiod in which the value of the MC ON/OFF signal is “0”, thereliability gradually increases from 0% to 100%. On the other hand,during a period in which the value of the MC ON/OFF signal is “1”, thereliability gradually decreases from 100% to 0%.

By considering the reliability in detection of the motion vector mv, inthe interpolation section 45, the imaging blur suppression processor 13,and the overdrive processor 10, it is set so that the degree of thevideo signal process rises as the reliability increases, and on theother hand, the degree of the video signal process falls off as thereliability decreases.

Concretely, the overdrive processor 10 sets so that as the reliabilityincreases, the degree of the overdrive process increases and, on theother hand, as the reliability decreases, the degree of the overdriveprocess decreases. Meanwhile, it is also possible to change the degreeof the overdrive process in accordance with the distance of theinterpolation position toward the video image of the closer originalframe which is set in each of interpolation frames by the interpolationsection 45 (to vary the degree of reduction in a motion blur and a holdblur in accordance with nonuniformity of the positions of interpolationframes (strength of a judder)), and to perform the overdrive process inconsideration of the reliability as well.

Further, the imaging blur suppression processor 13 sets so that thedegree of the imaging blur suppression process rises as the reliabilityincreases and, on the other hand, the degree of the imaging blursuppression process falls off as the reliability decreases. For example,like in the video signal processor 4C shown in FIG. 42 in the secondembodiment, the process amount in the imaging blur suppression processmay be changed in accordance with the distance of the interpolationposition toward a video image of the closer original frame ininterpolation frames, which is set by the interpolation section 45 (thedegree of reducing the imaging blur is changed according tononuniformity of positions of interpolation frames (strength of ajudder)) and, in addition, the imaging blur suppressing process may beperformed in consideration of such reliability.

Further, the interpolation section 45 changes the distance of settingthe interpolation position toward to a video image of the closeroriginal frame in each of interpolation frames in consideration ofreliability in detection of the motion vector mv. Thereby, nonuniformityof positions of interpolation frames (strength of a judder) can bechanged in consideration of reliability in detection of the motionvector mv.

Meanwhile, in the case of converting the frame rate of a video signal byadding M interpolation frames (where M is an integer of 1 or larger)obtained by interpolating video images of original frames between theoriginal frames neighboring along the time base using motioncompensation in place of the interpolation section 45, for example, aframe rate converting process in consideration of reliability may beperformed as shown in FIG. 46 (in the case of a 3:2 pulldown signal) andFIG. 47 (in the case of a 24 Hz film source signal).

Concretely, it may be set so that as the reliability increases, the gainmultiplied with motion vectors MV1 to MV3 at the time of frame rateconversion increases, and, on the other hand, as the reliabilitydecreases, the gain multiplied with the motion vectors MV1 to MV3 at thetime of frame rate conversion decreases.

In such a manner, in the embodiment, the video signal process in theinterpolation section 45, the imaging blur suppression processor 13, andthe overdrive processor 10 is performed in consideration of thereliability in detection of the motion vector mv by the motion vectordetector 44. It is set so that as the reliability increases, the degreeof the video signal process rises and, on the other hand, as thereliability decreases, the degree of the video signal process falls off.Consequently, in the case of performing the video signal process usingthe motion vector, even when the motion vector lies out of the motionvector search range (block matching range), the video signal processaccording to the detection precision of a motion vector can beperformed. Therefore, at the time of performing a predetermined videosignal process, deterioration in the picture quality due to the motionvector detection precision can be suppressed.

Fourth Embodiment

A fourth embodiment of the present invention will now be described.

FIG. 48 shows an example of the configuration of an image displayapparatus (a liquid crystal display 7) according to the embodiment. Inaddition, the same reference numerals are designated to the samecomponents as those in the foregoing embodiments and their descriptionwill not be repeated.

The liquid crystal display 7 displays a video image on the basis of avideo signal subjected to the video signal process in the video signalprocessor 4 (or any one of the vide signal processing apparatuses 4A to4D) described in the first to third embodiments, and is a hold-typedisplay apparatus. Concretely, the liquid crystal display 7 has thevideo signal processor 4 (4A to 4D), a liquid crystal display panel 70,a backlight driving unit 71, a backlight 72, a timing controller 73, agate driver 74, and a data driver 75.

The backlight 72 is a light source of emitting light to the liquidcrystal display panel 70 and includes, for example, a CCFL (Cold CathodeFluorescent Lamp) and an LED (Light Emitting Diode).

The liquid crystal display panel 70 modulates irradiation light from thebacklight 72 on the basis of a video signal. The liquid crystal displaypanel 70 includes a transmission-type liquid crystal layer (not shown),a pair of substrates (a TFT substrate and an opposite electrodesubstrate which are not shown) sandwiching the liquid crystal layer, andpolarizing plates (not shown) laminated on each of the TFT substrate andthe opposite electrode substrate on the side opposite to the liquidcrystal layer.

The data driver 75 supplies a drive voltage based on the video signal toeach of the pixel electrodes in a liquid crystal display panel 2. Thegate driver 74 line-sequentially drives the pixel electrodes in theliquid crystal display panel 2 along not-shown horizontal scan lines.The timing controller 73 controls the data driver 75 and the gate driver74 on the basis of the video signal supplied from the video signalprocessor 4 (4A to 4D). The backlight driving unit 71 controls theturn/on and turn/off operation of the backlight 72 (performs turn-ondriving on the backlight 72) on the basis of the video signal suppliedto the video signal processor 4 (4A to 4D).

The liquid crystal display 7 of the embodiment is constructed to performa black inserting process of inserting a black display area into adisplay screen of the liquid crystal display panel 2 in accordance withat least one of the video signal substance in an original frame andluminance of a viewing environment for user. Concretely, for example,the liquid crystal display 7 is constructed to perform the blackinserting process for inserting a black display area into the displayscreen in the liquid crystal display panel 2 when the video signal inthe original frame is a cinema signal (film signal). More concretely,the backlight driving unit 71 performs switching drive between turn-onand turn-off of the backlight 72 so that the process of inserting theblack display on the display screen in the liquid crystal display panel2 is performed. In addition, the backlight driving unit 71 determines,for example, whether the video signal in an original frame is a cinemasignal or not by using contents information of the original frameincluded in an EPG (Electronic Program Guide) or on the basis of theframe rate of the original frame.

The black inserting can be performed in the following methods. Forexample, the black inserting process is performed on the frame unitbasis as shown in FIGS. 49(A) and (B). For example, the black insertingprocess is performed by black insertion line unit composed of apredetermined number of horizontal scan lines in the original frame asshown in FIGS. 50(A) and (B). For example, as shown in FIGS. 51(A) and(B), the black inserting process is performed by a combination of theblack insertion line unit basis and the frame unit basis. In FIGS. 49 to51 (and FIGS. 52 to 55 described later), (A) shows the substance of thevideo image (original frames A to C and interpolation frames A′ to C′)in the liquid crystal display panel 2 (LCD), and (B) shows the light-onstate of the backlight 72. The horizontal axis in the diagram showstime.

In the case of the black inserting process on the frame unit basis shownin FIG. 49, an entire frame is lighted on or off, so that the holdimprovement effect increases. In the case of the black inserting processon the black insertion line unit basis shown in FIG. 50, displayluminance can be adjusted by setting a black insertion ratio which willbe described later, and the frame rate increases falsely. Consequently,as compared with the case of the frame unit basis, flicker isless-visible. In the case of the combination of the frame unit and theblack insertion line unit shown in FIG. 51, moving image responsebecomes the highest.

In addition, in the case including the black inserting process on theblack insertion line unit basis as shown in FIGS. 50 and 51, the blackinserting process may be performed by a plurality of black insertionlines away from each other. In such a configuration, adjustment of theblack insertion ratio and display luminance which will be describedbelow is facilitated.

Further, for example, as shown in FIGS. 52 to 55, when performing theblack inserting process, the backlight driving unit 71 may performswitching drive so that an areal ratio of a black display area in anentire display screen (=black insertion ratio) can be varied by changingthe thickness of a black insertion line (the number of horizontal scanlines constructing a black insertion line). In such a configuration, theeffect of reducing a hold blur is produced and the display luminance canbe adjusted.

Further, the backlight driving unit 71 may perform the switching driveso that the luminance of the black display region is varied whenperforming the black inserting process. In such a configuration, whilereducing a hold blur, the display luminance can be adjusted. Moreover,both of the black insertion ratio and luminance of the black displayarea may be varied.

In addition, in the case of making at least one of the black insertionratio and the luminance of the black display region variable, the blackinsertion ratio and the luminance of the black display region may bechanged in multiple stages or changed continuously. In the case ofmaking such changes, reduction in the hold blur and adjustment of thedisplay luminance is facilitated.

In such a manner, in the embodiment, the black inserting process ofinserting a black display area into a display screen in the liquidcrystal display panel 2 is performed in accordance with at least one ofthe substance of the video signal in an original frame and luminance ofthe viewing environment for user. Thus, a hold blur can be reducedaccording to the circumstances.

In addition, for example, as shown in FIG. 56, according to a luminancehistogram distribution in an original frame, whether the black insertingprocess is performed or not may be determined and the black insertionratio and the luminance of the black display area may be changed. Insuch a configuration, adjustments become possible. For example, it isdetermined that the black inserting process is performed in a case suchthat decrease in the display luminance is not conspicuous in a darkimage or the like, and by increasing the black insertion ratio ordecreasing the luminance of the black display area, priority is given tothe effect of reducing a hold blur.

Further, according to the magnitude of a motion vector in an originalframe detected by the motion vector detector 44 or the like, the blackinsertion ratio and the luminance of the black display area may bechanged. In such a configuration, for example, adjustment such assuppression of a judder can be performed by increasing the blackinsertion ratio or decreasing luminance of the black display area in thecase such that motion of a video image is large.

Further, for example, like in the liquid crystal display 7A shown inFIG. 57, by providing a brightness detector 76 for detecting brightnessof the user's viewing environment (constructed by, for example, anilluminance sensor) or the like, whether the black inserting process isperformed or not may be determined as described above or the blackinsertion ratio and the illuminance of the black display area may bechanged according to the detected brightness of the user's viewingenvironment. In such a configuration, there is a case such that decreasein the display luminance is not conspicuous depending on the brightnessof the viewing environment (for example, in the case where the viewingenvironment is a dark state). In such a case, the adjustment of placingpriority on the effect of reducing a hold blur can be performed bydetermining that the black inserting process is performed or increasingthe black insertion ratio or decreasing the luminance of the blackdisplay area.

In addition, in the case of providing the brightness detector 76,according to the detected brightness of the user's viewing environment,for example, the process amount of the imaging blur suppressing processby the imaging blur suppression processor 13 and degree of setting theinterpolation position nearer to a video image of a closer originalframe in each of the interpolation frames by the interpolation section45 may be changed. In the case of changing the process amount of theimaging blur suppressing process by the imaging blur suppressionprocessor 13, there is a case that an imaging blur is not conspicuousdepending on the brightness of the viewing environment (for example, inthe case where the viewing environment is a dark state). In such a case,the adjustment of decreasing the process amount of the imaging blursuppressing process can be performed. In the case of changing the degreeof setting the interpolation position nearer to a video image of acloser original frame by the interpolation section 45, when a judder isnot conspicuous depending on the brightness of the viewing environment(for example, in the case where the viewing environment is a darkstate), by setting the interpolation position nearer to the originalframe, a judder is left. In such a manner, for example, adjustment ofcreating realism peculiar to a movie is realized.

In addition, the embodiment has been described by the case where thehold-type image display apparatus is a liquid crystal display and theblack inserting process (blinking process) is performed by the switchdriving of the backlight driving unit 71. For example, in the case wherea display apparatus is a light-emitting display apparatus other than aliquid crystal display (such as an organic EL display apparatus), theblack inserting process may be performed by providing a black insertingprocessor (not shown) for performing a black inserting process on avideo signal of an original frame in the video signal processor, andperforming the video signal process by the black inserting processor,for example.

Meanwhile, in the embodiment, whether the black inserting process can beexecuted or not, a change in the black insertion ratio, a change inilluminance in the black display area, and the like can be set by anoperation of the user by providing, for example, predetermined operationmeans (setting means).

Further, the video signal processor in the embodiment is not limited tothe video signal processor 4 (or any one of the video signal processors4A to 4D) described in the first to third embodiments. Another videosignal processor may be employed as long as it performs a predeterminedvideo signal process on a plurality of original frames along the timebase.

Further, the series of processes (or a part of the processes) describedin the first to fourth embodiments can be executed by hardware orsoftware.

In this case, all of the video signal processors 4 and 4A to 4D, thebacklight driving unit 71, and the timing controller 73 or a part ofthem (for example, the imaging blur suppression processor 13 or thelike) described in the above-mentioned embodiments 1 to 4 can beconstructed by, for example, a computer as shown in FIG. 58.

In FIG. 58, a CPU (Central Processing Unit) 301 executes variousprocesses in accordance with a program recorded on a ROM (Read OnlyMemory) 302, or a program loaded from a storage 308 to a RAM (RandomAccess memory) 303. In the RAM 303, data or the like necessary for theCPU 301 to execute various processes is also properly stored.

The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus304. To the bus 304, an input/output interface 305 is also connected.

To the input/output interface 305, an input unit 306 including akeyboard, a mouse, and the like, an output unit 307 such as a display, astorage 308 constructed by a hard disk or the like, and a communicationunit 309 including a modem, a terminal adapter, and the like areconnected. The communication unit 309 performs communication processwith other devices via networks including the Internet.

As necessary, a drive 310 is also connected to the input/outputinterface 305. A removable recording medium 311 such as a magnetic disk,an optical disk, a magneto-optic disk, or a semiconductor memory isproperly mounted on the input/output interface 305. A computer programread from the removable recording medium 311 is installed in the storage308 as necessary.

In the case of executing a series of processes by software, a programconstructing the software is installed from a network or a recordingmedium to, for example, a computer assembled in dedicated hardware or ageneral personal computer or the like capable of executing variousfunctions by installing various programs.

The recording medium including such a program is not limited to theremovable recording medium (package medium) 211 such as a magnetic disk(including a floppy disk), an optical disk (including a CD-ROM (CompactDisk-Read Only memory) and a DVD (Digital Versatile Disk)), amagneto-optic disk (including an MD (Mini-Disk)), or a semiconductormemory, as shown in FIG. 58. It may be the ROM 302 in which a program isrecorded, a hard disk included in the storage 308, or the like which isprovided to the user in a state where it is pre-assembled in theapparatus body.

Meanwhile, in the specification, the steps describing a program to berecorded on a recording medium include, obviously, not only processesperformed in time series in the order, but also processes which are notalways performed in time series but are executed in parallel orindividually.

Further, as described above, in the specification, the system refers toan entire apparatus constructed by a plurality of processing apparatusesand processors.

Further, the configurations and the like described in the foregoingembodiments and modified examples are not limited to the above-describedcombinations but can be arbitrarily combined.

1. An image display apparatus characterized by comprising: motion vector detecting means for detecting a motion vector in a plurality of original frames along a time base; video signal processing means for performing, by using the detected motion vector, a predetermined video signal process to improve picture quality on the plurality of original frames; and display means for displaying a video image on the basis of a video signal subjected to the video signal process, wherein the video signal processing means performs the video signal process so that a degree of the video signal process rises as reliability in detection of the motion vector by the motion vector detecting means increases and, on the other hand, a degree of the video signal process falls off as the reliability decreases.
 2. The image display apparatus according to claim 1, characterized in that the video signal processing means includes imaging blur suppression processing means for performing, by using the detected motion vector, an imaging blur suppressing process to suppress picture quality deterioration due to an imaging blur included in the original frame.
 3. The image display apparatus according to claim 2, characterized in that the imaging blur suppression processing means performs the video signal process so that a degree of the imaging blur suppression process rises as the reliability increases and, on the other hand, a degree of the imaging blur suppression process falls off as the reliability decreases.
 4. The image display apparatus according to claim 1, characterized in that the video signal processing means includes overdrive process means for performing an overdrive process on a video signal of the original frame by using the detected motion vector.
 5. The image display apparatus according to claim 4, characterized in that the overdrive processing means performs the video signal process so that a degree of the overdrive process rises as the reliability increases and, on the other hand, a degree of the overdrive process falls off as the reliability decreases.
 6. The image display apparatus according to claim 1, characterized in that the video signal processing means includes frame rate converting means for converting frame rate of a video signal by adding M (M: integer of 1 or larger) interpolation frames into between original frames neighboring each other along the time base, the interpolation frames being obtained from video images in the original frames by using motion compensation.
 7. The image display apparatus according to claim 6, characterized in that the frame rate converting means performs the video signal process so that a gain with which the motion vector is multiplied at the time of the frame rate conversion rises as the reliability increases and, on the other hand, the gain falls off as the reliability decreases.
 8. The image display apparatus according to any one of claims 1 to 7, characterized in that the value of the reliability increases gradually in a period in which the motion vector is detected and, on the other hand, the value of the reliability decreases gradually in a period in which the motion vector is not detected.
 9. A video signal processor characterized by comprising: motion vector detecting means for detecting a motion vector in a plurality of original frames along a time base; and video signal processing means for performing, by using the detected motion vector, a predetermined video signal process to improve picture quality on the plurality of original frames, wherein the video signal processing means performs the video signal process so that a degree of the video signal process rises as reliability in detection of the motion vector by the motion vector detecting means increases and, on the other hand, a degree of the video signal process falls off as the reliability decreases.
 10. A video signal processing method characterized by comprising the steps of: detecting a motion vector in a plurality of original frames along a time base; and performing, by using the detected motion vector, a predetermined video signal process to improve picture quality on the plurality of original frames, wherein the video signal process is performed so that a degree of the video signal process rises as reliability in detection of the motion vector increases and, on the other hand, a degree of the video signal process falls off as the reliability decreases. 